2024-02-05T07:32:41.442 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-05T07:32:41.443 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi --description orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} --archive /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546764 --owner scheduled_gabrioux@teuthology -- /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546764/orig.config.yaml 2024-02-05T07:32:41.470 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-05T07:32:41.542 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546764 branch: wip-guits-testing-3-2024-02-04-2254 description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} email: gabrioux@ibm.com first_in_suite: false job_id: '7546764' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-testing-3-2024-02-04-2254 ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms bind msgr1: false ms bind msgr2: true ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 extra_packages: - cephadm selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-testing-3-2024-02-04-2254 sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 7317 sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-testing-3-2024-02-04-2254 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_164d3be4a527be775f4d3c343ea591122f1b8921/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 targets: smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDHqdPFifKEZTDIdgd+pGQ9OsOQQVTz8Qu79nVU1HggFfWWCt+N00mHMx5wMV+qaCuh/CLE3AjlTQGpFb/6SSEE= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLl+NCAQSYeiV9rpCABJcuPqtPiDOjSZFuxQWYUIqg3f6zeHNOEe6AFKT8u7BgDJ1PDrQiXebd8ofmOBQLObPzQ= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-02-05_05:57:01 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-05T07:32:41.542 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_164d3be4a527be775f4d3c343ea591122f1b8921/qa; will attempt to use it 2024-02-05T07:32:41.543 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_164d3be4a527be775f4d3c343ea591122f1b8921/qa/tasks 2024-02-05T07:32:41.543 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-05T07:32:41.544 INFO:teuthology.task.internal:Checking packages... 2024-02-05T07:32:41.565 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '164d3be4a527be775f4d3c343ea591122f1b8921' 2024-02-05T07:32:41.565 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-05T07:32:41.565 INFO:teuthology.packaging:ref: None 2024-02-05T07:32:41.565 INFO:teuthology.packaging:tag: None 2024-02-05T07:32:41.565 INFO:teuthology.packaging:branch: wip-guits-testing-3-2024-02-04-2254 2024-02-05T07:32:41.565 INFO:teuthology.packaging:sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:32:41.565 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-testing-3-2024-02-04-2254 2024-02-05T07:32:41.708 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1241.g164d3be4 2024-02-05T07:32:41.709 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-05T07:32:41.716 INFO:teuthology.task.internal:no buildpackages task found 2024-02-05T07:32:41.716 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-05T07:32:41.722 INFO:teuthology.task.internal:Saving configuration 2024-02-05T07:32:41.733 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-05T07:32:41.739 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-05T07:32:41.761 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546764', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-05 07:26:53.390231', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDHqdPFifKEZTDIdgd+pGQ9OsOQQVTz8Qu79nVU1HggFfWWCt+N00mHMx5wMV+qaCuh/CLE3AjlTQGpFb/6SSEE='} 2024-02-05T07:32:41.778 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546764', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-05 07:26:53.389042', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLl+NCAQSYeiV9rpCABJcuPqtPiDOjSZFuxQWYUIqg3f6zeHNOEe6AFKT8u7BgDJ1PDrQiXebd8ofmOBQLObPzQ='} 2024-02-05T07:32:41.778 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-05T07:32:41.784 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-05T07:32:41.785 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-02-05T07:32:41.785 INFO:teuthology.run_tasks:Running task console_log... 2024-02-05T07:32:41.856 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd8744600d0>, signals=[15]) 2024-02-05T07:32:41.856 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-05T07:32:41.863 INFO:teuthology.task.internal:Opening connections... 2024-02-05T07:32:41.863 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2024-02-05T07:32:41.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:32:41.954 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-02-05T07:32:41.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:32:42.041 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-05T07:32:42.048 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2024-02-05T07:32:42.066 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2024-02-05T07:32:42.066 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2024-02-05T07:32:42.123 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2024-02-05T07:32:42.123 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="9" 2024-02-05T07:32:42.123 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2024-02-05T07:32:42.123 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2024-02-05T07:32:42.123 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="9" 2024-02-05T07:32:42.123 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el9" 2024-02-05T07:32:42.123 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-05T07:32:42.123 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2024-02-05T07:32:42.124 INFO:teuthology.orchestra.run.smithi131.stdout:LOGO="fedora-logo-icon" 2024-02-05T07:32:42.124 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-05T07:32:42.124 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2024-02-05T07:32:42.124 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-05T07:32:42.124 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-05T07:32:42.124 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-05T07:32:42.124 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2024-02-05T07:32:42.148 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-02-05T07:32:42.165 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-02-05T07:32:42.165 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-02-05T07:32:42.221 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2024-02-05T07:32:42.222 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2024-02-05T07:32:42.222 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2024-02-05T07:32:42.222 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2024-02-05T07:32:42.222 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2024-02-05T07:32:42.222 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2024-02-05T07:32:42.222 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-05T07:32:42.222 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2024-02-05T07:32:42.222 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2024-02-05T07:32:42.222 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-05T07:32:42.222 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2024-02-05T07:32:42.222 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-05T07:32:42.222 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-05T07:32:42.222 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-05T07:32:42.222 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-02-05T07:32:42.239 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-05T07:32:42.248 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-05T07:32:42.253 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-05T07:32:42.253 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2024-02-05T07:32:42.255 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-02-05T07:32:42.277 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-05T07:32:42.283 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-05T07:32:42.283 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2024-02-05T07:32:42.311 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-02-05T07:32:42.544 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-05T07:32:42.601 INFO:teuthology.run_tasks:Running task kernel... 2024-02-05T07:32:42.616 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-05T07:32:42.616 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-05T07:32:42.616 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-05T07:32:42.617 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-05T07:32:42.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T07:32:42.633 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-02-05T07:32:42.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T07:32:42.634 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-02-05T07:32:42.690 INFO:teuthology.orchestra.run.smithi131.stdout:5.14.0-361.el9.x86_64 2024-02-05T07:32:42.690 INFO:teuthology.task.kernel:Running kernel on smithi131: 5.14.0-361.el9.x86_64 2024-02-05T07:32:42.691 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum install -y kernel 2024-02-05T07:32:42.692 INFO:teuthology.orchestra.run.smithi157.stdout:5.14.0-361.el9.x86_64 2024-02-05T07:32:42.692 INFO:teuthology.task.kernel:Running kernel on smithi157: 5.14.0-361.el9.x86_64 2024-02-05T07:32:42.692 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum install -y kernel 2024-02-05T07:32:44.813 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - AppStream 13 MB/s | 6.6 MB 00:00 2024-02-05T07:32:44.845 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS-9 - AppStream 13 MB/s | 6.6 MB 00:00 2024-02-05T07:32:46.205 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - BaseOS 6.0 MB/s | 2.0 MB 00:00 2024-02-05T07:32:46.262 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS-9 - BaseOS 6.5 MB/s | 2.0 MB 00:00 2024-02-05T07:32:48.079 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.0 MB 00:01 2024-02-05T07:32:48.470 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-02-05T07:32:57.337 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 2.4 MB/s | 19 MB 00:07 2024-02-05T07:33:06.428 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-02-05T07:33:13.411 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 483 kB/s | 5.9 MB 00:12 2024-02-05T07:33:15.221 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 15 kB 00:00 2024-02-05T07:33:17.638 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 20 MB 00:02 2024-02-05T07:33:20.560 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 574 kB/s | 5.9 MB 00:10 2024-02-05T07:33:22.053 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 15 kB 00:00 2024-02-05T07:33:22.655 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-02-05T07:33:23.346 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 17 MB/s | 20 MB 00:01 2024-02-05T07:33:24.862 INFO:teuthology.orchestra.run.smithi131.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-05T07:33:24.862 INFO:teuthology.orchestra.run.smithi131.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-05T07:33:24.958 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:33:24.968 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:33:24.968 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-02-05T07:33:24.968 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:33:24.968 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:33:24.968 INFO:teuthology.orchestra.run.smithi131.stdout: kernel x86_64 5.14.0-416.el9 CentOS-BaseOS 6.0 M 2024-02-05T07:33:24.968 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-02-05T07:33:24.968 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-core x86_64 5.14.0-416.el9 CentOS-BaseOS 21 M 2024-02-05T07:33:24.968 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-modules x86_64 5.14.0-416.el9 CentOS-BaseOS 39 M 2024-02-05T07:33:24.968 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-modules-core x86_64 5.14.0-416.el9 CentOS-BaseOS 33 M 2024-02-05T07:33:24.968 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:33:24.968 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:33:24.968 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:33:24.968 INFO:teuthology.orchestra.run.smithi131.stdout:Install 4 Packages 2024-02-05T07:33:24.968 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:33:24.969 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 99 M 2024-02-05T07:33:24.969 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 122 M 2024-02-05T07:33:24.969 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:33:25.356 INFO:teuthology.orchestra.run.smithi131.stdout:(1/4): kernel-5.14.0-416.el9.x86_64.rpm 16 MB/s | 6.0 MB 00:00 2024-02-05T07:33:26.006 INFO:teuthology.orchestra.run.smithi131.stdout:(2/4): kernel-core-5.14.0-416.el9.x86_64.rpm 20 MB/s | 21 MB 00:01 2024-02-05T07:33:26.657 INFO:teuthology.orchestra.run.smithi131.stdout:(3/4): kernel-modules-5.14.0-416.el9.x86_64.rpm 23 MB/s | 39 MB 00:01 2024-02-05T07:33:27.065 INFO:teuthology.orchestra.run.smithi131.stdout:(4/4): kernel-modules-core-5.14.0-416.el9.x86_6 20 MB/s | 33 MB 00:01 2024-02-05T07:33:27.069 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:33:27.069 INFO:teuthology.orchestra.run.smithi131.stdout:Total 47 MB/s | 99 MB 00:02 2024-02-05T07:33:27.069 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:33:27.469 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:33:27.469 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:33:28.387 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 6.3 kB/s | 1.7 kB 00:00 2024-02-05T07:33:28.499 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:33:28.500 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:33:30.610 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-05T07:33:30.610 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-05T07:33:30.704 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:33:30.710 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:33:30.710 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-02-05T07:33:30.710 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:33:30.710 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:33:30.710 INFO:teuthology.orchestra.run.smithi157.stdout: kernel x86_64 5.14.0-416.el9 CentOS-BaseOS 6.0 M 2024-02-05T07:33:30.710 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-02-05T07:33:30.710 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-core x86_64 5.14.0-416.el9 CentOS-BaseOS 21 M 2024-02-05T07:33:30.710 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules x86_64 5.14.0-416.el9 CentOS-BaseOS 39 M 2024-02-05T07:33:30.710 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules-core x86_64 5.14.0-416.el9 CentOS-BaseOS 33 M 2024-02-05T07:33:30.711 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:33:30.711 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:33:30.711 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:33:30.711 INFO:teuthology.orchestra.run.smithi157.stdout:Install 4 Packages 2024-02-05T07:33:30.711 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:33:30.711 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 99 M 2024-02-05T07:33:30.711 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 122 M 2024-02-05T07:33:30.711 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:33:30.933 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:33:31.097 INFO:teuthology.orchestra.run.smithi157.stdout:(1/4): kernel-5.14.0-416.el9.x86_64.rpm 16 MB/s | 6.0 MB 00:00 2024-02-05T07:33:31.593 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-modules-core-5.14.0-416.el9.x86_64 1/4 2024-02-05T07:33:31.672 INFO:teuthology.orchestra.run.smithi157.stdout:(2/4): kernel-core-5.14.0-416.el9.x86_64.rpm 22 MB/s | 21 MB 00:00 2024-02-05T07:33:31.728 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T07:33:32.130 INFO:teuthology.orchestra.run.smithi157.stdout:(3/4): kernel-modules-core-5.14.0-416.el9.x86_6 32 MB/s | 33 MB 00:01 2024-02-05T07:33:32.689 INFO:teuthology.orchestra.run.smithi157.stdout:(4/4): kernel-modules-5.14.0-416.el9.x86_64.rpm 20 MB/s | 39 MB 00:01 2024-02-05T07:33:32.692 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:33:32.692 INFO:teuthology.orchestra.run.smithi157.stdout:Total 50 MB/s | 99 MB 00:01 2024-02-05T07:33:32.693 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:33:33.090 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:33:33.090 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:33:34.137 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:33:34.138 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:33:34.499 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T07:33:34.790 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T07:33:36.551 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:33:37.212 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-modules-core-5.14.0-416.el9.x86_64 1/4 2024-02-05T07:33:37.356 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T07:33:39.473 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T07:33:39.586 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:33:40.146 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T07:33:40.335 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T07:33:44.324 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-modules-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:33:45.083 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T07:33:45.271 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:33:50.015 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:34:13.636 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:34:13.637 INFO:teuthology.orchestra.run.smithi131.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-05T07:34:13.637 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:34:13.706 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-modules-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:34:14.135 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:34:14.135 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-5.14.0-416.el9.x86_64 1/4 2024-02-05T07:34:14.136 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T07:34:14.136 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T07:34:14.553 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-modules-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:34:14.553 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:34:14.553 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:34:14.553 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-5.14.0-416.el9.x86_64 2024-02-05T07:34:14.553 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-core-5.14.0-416.el9.x86_64 2024-02-05T07:34:14.553 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-modules-5.14.0-416.el9.x86_64 2024-02-05T07:34:14.554 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-modules-core-5.14.0-416.el9.x86_64 2024-02-05T07:34:14.554 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:34:14.554 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:34:15.048 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q kernel | sort -rV | head -n 1 2024-02-05T07:34:15.178 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-5.14.0-416.el9.x86_64 2024-02-05T07:34:15.178 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-416.el9.x86_64 2024-02-05T07:34:15.179 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-416.el9.x86_64 2024-02-05T07:34:15.179 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-05T07:34:15.179 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-05T07:34:15.179 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-05T07:34:15.179 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi131.front.sepia.ceph.com, path=None, version=distro) 2024-02-05T07:34:15.179 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q kernel | sort -rV | head -n 1 2024-02-05T07:34:15.285 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-5.14.0-416.el9.x86_64 2024-02-05T07:34:15.286 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-416.el9.x86_64 2024-02-05T07:34:15.286 DEBUG:teuthology.orchestra.run.smithi131:> sudo rpm -qi grub2-tools 2024-02-05T07:34:15.328 INFO:teuthology.orchestra.run.smithi131.stdout:Name : grub2-tools 2024-02-05T07:34:15.328 INFO:teuthology.orchestra.run.smithi131.stdout:Epoch : 1 2024-02-05T07:34:15.328 INFO:teuthology.orchestra.run.smithi131.stdout:Version : 2.06 2024-02-05T07:34:15.328 INFO:teuthology.orchestra.run.smithi131.stdout:Release : 68.el9 2024-02-05T07:34:15.328 INFO:teuthology.orchestra.run.smithi131.stdout:Architecture: x86_64 2024-02-05T07:34:15.328 INFO:teuthology.orchestra.run.smithi131.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-05T07:34:15.328 INFO:teuthology.orchestra.run.smithi131.stdout:Group : Unspecified 2024-02-05T07:34:15.328 INFO:teuthology.orchestra.run.smithi131.stdout:Size : 8264254 2024-02-05T07:34:15.328 INFO:teuthology.orchestra.run.smithi131.stdout:License : GPLv3+ 2024-02-05T07:34:15.328 INFO:teuthology.orchestra.run.smithi131.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-05T07:34:15.329 INFO:teuthology.orchestra.run.smithi131.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-05T07:34:15.329 INFO:teuthology.orchestra.run.smithi131.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-05T07:34:15.329 INFO:teuthology.orchestra.run.smithi131.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-05T07:34:15.329 INFO:teuthology.orchestra.run.smithi131.stdout:Packager : builder@centos.org 2024-02-05T07:34:15.329 INFO:teuthology.orchestra.run.smithi131.stdout:Vendor : CentOS 2024-02-05T07:34:15.329 INFO:teuthology.orchestra.run.smithi131.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-05T07:34:15.329 INFO:teuthology.orchestra.run.smithi131.stdout:Summary : Support tools for GRUB. 2024-02-05T07:34:15.329 INFO:teuthology.orchestra.run.smithi131.stdout:Description : 2024-02-05T07:34:15.329 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:34:15.329 INFO:teuthology.orchestra.run.smithi131.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-05T07:34:15.329 INFO:teuthology.orchestra.run.smithi131.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-05T07:34:15.329 INFO:teuthology.orchestra.run.smithi131.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-05T07:34:15.329 INFO:teuthology.orchestra.run.smithi131.stdout:hardware devices. 2024-02-05T07:34:15.329 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:34:15.330 INFO:teuthology.orchestra.run.smithi131.stdout:This subpackage provides tools for support of all platforms. 2024-02-05T07:34:15.330 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-05T07:34:15.331 INFO:teuthology.task.kernel:Updating grub on smithi131 to boot 5.14.0-416.el9.x86_64 2024-02-05T07:34:15.331 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-05T07:34:15.920 INFO:teuthology.orchestra.run.smithi131.stderr:Generating grub configuration file ... 2024-02-05T07:34:17.310 INFO:teuthology.orchestra.run.smithi131.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-05T07:34:17.344 INFO:teuthology.orchestra.run.smithi131.stderr:done 2024-02-05T07:34:17.346 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-02-05T07:34:17.361 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.vrGAnxF2Y6 2024-02-05T07:34:17.362 DEBUG:teuthology.orchestra.run.smithi131:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.vrGAnxF2Y6 2024-02-05T07:34:17.430 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.vrGAnxF2Y6 2024-02-05T07:34:17.563 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.vrGAnxF2Y6 is 6KB 2024-02-05T07:34:17.612 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.vrGAnxF2Y6 2024-02-05T07:34:17.626 DEBUG:teuthology.orchestra.run.smithi131:> sudo /bin/ls /boot/loader/entries || true 2024-02-05T07:34:17.695 INFO:teuthology.orchestra.run.smithi131.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-05T07:34:17.695 INFO:teuthology.orchestra.run.smithi131.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-05T07:34:17.695 INFO:teuthology.orchestra.run.smithi131.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-05T07:34:17.695 INFO:teuthology.orchestra.run.smithi131.stdout:15949e3c88704300882482eddfd7baae-5.14.0-416.el9.x86_64.conf 2024-02-05T07:34:17.696 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-416.el9.x86_64 2024-02-05T07:34:17.858 DEBUG:teuthology.orchestra.run.smithi131:> sudo shutdown -r now 2024-02-05T07:34:19.105 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:34:19.106 INFO:teuthology.orchestra.run.smithi157.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-05T07:34:19.106 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:34:19.177 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:34:19.640 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:34:19.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-5.14.0-416.el9.x86_64 1/4 2024-02-05T07:34:19.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T07:34:19.640 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T07:34:20.085 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-modules-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:34:20.085 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:34:20.085 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:34:20.085 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-5.14.0-416.el9.x86_64 2024-02-05T07:34:20.085 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-core-5.14.0-416.el9.x86_64 2024-02-05T07:34:20.085 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules-5.14.0-416.el9.x86_64 2024-02-05T07:34:20.085 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules-core-5.14.0-416.el9.x86_64 2024-02-05T07:34:20.085 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:34:20.085 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:34:20.602 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q kernel | sort -rV | head -n 1 2024-02-05T07:34:20.732 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-5.14.0-416.el9.x86_64 2024-02-05T07:34:20.732 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-416.el9.x86_64 2024-02-05T07:34:20.732 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-416.el9.x86_64 2024-02-05T07:34:20.732 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-05T07:34:20.733 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-05T07:34:20.733 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-05T07:34:20.733 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi157.front.sepia.ceph.com, path=None, version=distro) 2024-02-05T07:34:20.733 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q kernel | sort -rV | head -n 1 2024-02-05T07:34:20.841 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-5.14.0-416.el9.x86_64 2024-02-05T07:34:20.841 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-416.el9.x86_64 2024-02-05T07:34:20.841 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -qi grub2-tools 2024-02-05T07:34:20.874 INFO:teuthology.orchestra.run.smithi157.stdout:Name : grub2-tools 2024-02-05T07:34:20.874 INFO:teuthology.orchestra.run.smithi157.stdout:Epoch : 1 2024-02-05T07:34:20.874 INFO:teuthology.orchestra.run.smithi157.stdout:Version : 2.06 2024-02-05T07:34:20.874 INFO:teuthology.orchestra.run.smithi157.stdout:Release : 68.el9 2024-02-05T07:34:20.874 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: x86_64 2024-02-05T07:34:20.875 INFO:teuthology.orchestra.run.smithi157.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-05T07:34:20.875 INFO:teuthology.orchestra.run.smithi157.stdout:Group : Unspecified 2024-02-05T07:34:20.875 INFO:teuthology.orchestra.run.smithi157.stdout:Size : 8264254 2024-02-05T07:34:20.875 INFO:teuthology.orchestra.run.smithi157.stdout:License : GPLv3+ 2024-02-05T07:34:20.875 INFO:teuthology.orchestra.run.smithi157.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-05T07:34:20.875 INFO:teuthology.orchestra.run.smithi157.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-05T07:34:20.875 INFO:teuthology.orchestra.run.smithi157.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-05T07:34:20.875 INFO:teuthology.orchestra.run.smithi157.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-05T07:34:20.875 INFO:teuthology.orchestra.run.smithi157.stdout:Packager : builder@centos.org 2024-02-05T07:34:20.875 INFO:teuthology.orchestra.run.smithi157.stdout:Vendor : CentOS 2024-02-05T07:34:20.875 INFO:teuthology.orchestra.run.smithi157.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-05T07:34:20.875 INFO:teuthology.orchestra.run.smithi157.stdout:Summary : Support tools for GRUB. 2024-02-05T07:34:20.875 INFO:teuthology.orchestra.run.smithi157.stdout:Description : 2024-02-05T07:34:20.875 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:34:20.875 INFO:teuthology.orchestra.run.smithi157.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-05T07:34:20.875 INFO:teuthology.orchestra.run.smithi157.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-05T07:34:20.876 INFO:teuthology.orchestra.run.smithi157.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-05T07:34:20.876 INFO:teuthology.orchestra.run.smithi157.stdout:hardware devices. 2024-02-05T07:34:20.876 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:34:20.876 INFO:teuthology.orchestra.run.smithi157.stdout:This subpackage provides tools for support of all platforms. 2024-02-05T07:34:20.877 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-05T07:34:20.877 INFO:teuthology.task.kernel:Updating grub on smithi157 to boot 5.14.0-416.el9.x86_64 2024-02-05T07:34:20.877 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-05T07:34:21.532 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2024-02-05T07:34:22.995 INFO:teuthology.orchestra.run.smithi157.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-05T07:34:23.029 INFO:teuthology.orchestra.run.smithi157.stderr:done 2024-02-05T07:34:23.032 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-02-05T07:34:23.047 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.bOZgNcC1VM 2024-02-05T07:34:23.047 DEBUG:teuthology.orchestra.run.smithi157:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bOZgNcC1VM 2024-02-05T07:34:23.115 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.bOZgNcC1VM 2024-02-05T07:34:23.249 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.bOZgNcC1VM is 6KB 2024-02-05T07:34:23.269 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.bOZgNcC1VM 2024-02-05T07:34:23.283 DEBUG:teuthology.orchestra.run.smithi157:> sudo /bin/ls /boot/loader/entries || true 2024-02-05T07:34:23.347 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-05T07:34:23.348 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-05T07:34:23.348 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-05T07:34:23.348 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-5.14.0-416.el9.x86_64.conf 2024-02-05T07:34:23.349 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-416.el9.x86_64 2024-02-05T07:34:23.516 DEBUG:teuthology.orchestra.run.smithi157:> sudo shutdown -r now 2024-02-05T07:34:47.875 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-05T07:34:47.876 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-02-05T07:34:47.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:34:53.522 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-05T07:34:53.522 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-02-05T07:34:53.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:35:11.932 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-02-05T07:35:20.940 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-02-05T07:35:20.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:35:24.000 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-02-05T07:35:36.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-02-05T07:35:36.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:35:39.072 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-02-05T07:35:47.888 DEBUG:teuthology.orchestra.remote:timed out 2024-02-05T07:35:54.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-02-05T07:35:54.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:35:56.891 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-02-05T07:35:56.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:35:58.311 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-02-05T07:35:58.783 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-02-05T07:35:58.783 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-416.el9.x86_64"... 2024-02-05T07:35:58.783 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-02-05T07:35:58.797 INFO:teuthology.orchestra.run.smithi131.stdout:5.14.0-416.el9.x86_64 2024-02-05T07:35:58.798 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-416.el9.x86_64 vs 5.14.0-416.el9.x86_64 2024-02-05T07:35:58.798 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-05T07:35:58.798 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-05T07:35:59.799 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-05T07:35:59.799 DEBUG:teuthology.orchestra.run.smithi131:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-05T07:35:59.880 INFO:teuthology.orchestra.run.smithi131.stdout:ttyS1 2024-02-05T07:35:59.914 DEBUG:teuthology.parallel:result is None 2024-02-05T07:36:12.608 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-02-05T07:36:30.609 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-02-05T07:36:30.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:36:30.974 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-02-05T07:36:31.466 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-02-05T07:36:31.466 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-416.el9.x86_64"... 2024-02-05T07:36:31.466 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-02-05T07:36:31.481 INFO:teuthology.orchestra.run.smithi157.stdout:5.14.0-416.el9.x86_64 2024-02-05T07:36:31.481 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-416.el9.x86_64 vs 5.14.0-416.el9.x86_64 2024-02-05T07:36:31.481 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-05T07:36:31.482 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-05T07:36:32.483 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-05T07:36:32.483 DEBUG:teuthology.orchestra.run.smithi157:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-05T07:36:32.557 INFO:teuthology.orchestra.run.smithi157.stdout:ttyS1 2024-02-05T07:36:32.587 DEBUG:teuthology.parallel:result is None 2024-02-05T07:36:32.587 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-05T07:36:32.595 INFO:teuthology.task.internal:Creating test directory... 2024-02-05T07:36:32.595 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-05T07:36:32.599 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-05T07:36:32.618 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-05T07:36:32.628 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-05T07:36:32.638 INFO:teuthology.task.internal:Creating archive directory... 2024-02-05T07:36:32.639 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-05T07:36:32.661 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-05T07:36:32.761 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-05T07:36:32.768 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-05T07:36:32.768 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-05T07:36:32.771 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-05T07:36:32.811 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-05T07:36:32.822 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-05T07:36:32.825 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-05T07:36:32.835 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-05T07:36:32.837 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-05T07:36:32.845 INFO:teuthology.task.internal:Configuring sudo... 2024-02-05T07:36:32.845 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-05T07:36:32.869 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-05T07:36:32.897 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-05T07:36:32.907 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-05T07:36:32.907 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-05T07:36:32.940 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-05T07:36:32.955 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-05T07:36:33.042 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-05T07:36:33.086 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-05T07:36:33.170 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-05T07:36:33.212 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:36:33.212 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-05T07:36:33.280 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-05T07:36:33.324 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-05T07:36:33.376 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-05T07:36:33.458 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-05T07:36:33.500 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-02-05T07:36:33.500 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-05T07:36:33.567 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2024-02-05T07:36:33.570 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-02-05T07:36:33.612 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-05T07:36:33.647 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-05T07:36:34.057 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-05T07:36:34.066 INFO:teuthology.task.internal:Starting timer... 2024-02-05T07:36:34.066 INFO:teuthology.run_tasks:Running task pcp... 2024-02-05T07:36:34.084 INFO:teuthology.run_tasks:Running task selinux... 2024-02-05T07:36:34.092 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-05T07:36:34.093 DEBUG:teuthology.orchestra.run.smithi131:> sudo service auditd rotate 2024-02-05T07:36:34.160 INFO:teuthology.orchestra.run.smithi131.stdout:Rotating logs: 2024-02-05T07:36:34.162 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2024-02-05T07:36:34.210 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2024-02-05T07:36:34.211 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-05T07:36:34.212 DEBUG:teuthology.orchestra.run.smithi131:> /usr/sbin/getenforce 2024-02-05T07:36:34.236 INFO:teuthology.orchestra.run.smithi131.stdout:Permissive 2024-02-05T07:36:34.236 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2024-02-05T07:36:34.263 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2024-02-05T07:36:34.263 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi131.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2024-02-05T07:36:34.264 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-05T07:36:34.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T07:36:34.305 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-05T07:36:34.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T07:36:34.332 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-05T07:36:34.333 DEBUG:teuthology.orchestra.run.smithi131:> sudo /usr/sbin/setenforce permissive 2024-02-05T07:36:34.379 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2024-02-05T07:36:34.415 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-05T07:36:34.425 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-05T07:36:34.447 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-05T07:36:34.448 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi131.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-05T07:40:40.096 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi131.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-02-05T07:40:40.097 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-02-05T07:40:40.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:40:40.189 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-02-05T07:40:40.267 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-02-05T07:40:40.268 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-02-05T07:40:40.268 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:40:40.356 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-02-05T07:40:40.446 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-02-05T07:40:40.446 INFO:teuthology.run_tasks:Running task clock... 2024-02-05T07:40:40.460 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-05T07:40:40.460 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-05T07:40:40.461 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-05T07:40:40.463 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-05T07:40:40.463 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-05T07:40:40.492 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-05T07:40:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-05T07:40:40.518 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-05T07:40:40.534 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-05T07:40:40.538 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: ntpd: command not found 2024-02-05T07:40:40.552 INFO:teuthology.orchestra.run.smithi131.stdout:506 Cannot talk to daemon 2024-02-05T07:40:40.563 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2024-02-05T07:40:40.568 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-05T07:40:40.577 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2024-02-05T07:40:40.584 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-05T07:40:40.594 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-05T07:40:40.611 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-05T07:40:40.635 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-02-05T07:40:40.638 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-05T07:40:40.638 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-02-05T07:40:40.638 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T07:40:40.639 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T07:40:40.639 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T07:40:40.639 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T07:40:40.664 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-02-05T07:40:40.667 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-05T07:40:40.668 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-02-05T07:40:40.668 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T07:40:40.668 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T07:40:40.668 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T07:40:40.668 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T07:40:40.669 INFO:teuthology.run_tasks:Running task pexec... 2024-02-05T07:40:40.682 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-05T07:40:40.682 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi131.front.sepia.ceph.com 2024-02-05T07:40:40.682 DEBUG:teuthology.orchestra.run.smithi131:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-05T07:40:40.683 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi157.front.sepia.ceph.com 2024-02-05T07:40:40.683 DEBUG:teuthology.orchestra.run.smithi157:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-05T07:40:41.265 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:47 ago on Mon 05 Feb 2024 07:39:54 AM UTC. 2024-02-05T07:40:41.293 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:48 ago on Mon 05 Feb 2024 07:39:53 AM UTC. 2024-02-05T07:40:41.417 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:40:41.419 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:40:41.419 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-02-05T07:40:41.419 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:40:41.419 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:40:41.419 INFO:teuthology.orchestra.run.smithi131.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-05T07:40:41.419 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:40:41.419 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:40:41.419 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:40:41.419 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-02-05T07:40:41.419 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:40:41.420 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 3.1 M 2024-02-05T07:40:41.420 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 9.8 M 2024-02-05T07:40:41.420 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:40:41.444 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:40:41.453 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:40:41.453 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-02-05T07:40:41.453 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:40:41.453 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:40:41.454 INFO:teuthology.orchestra.run.smithi157.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-05T07:40:41.454 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:40:41.454 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:40:41.454 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:40:41.454 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-02-05T07:40:41.454 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:40:41.454 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 3.1 M 2024-02-05T07:40:41.455 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 9.8 M 2024-02-05T07:40:41.455 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:40:41.662 INFO:teuthology.orchestra.run.smithi131.stdout:runc-1.1.12-1.el9.x86_64.rpm 13 MB/s | 3.1 MB 00:00 2024-02-05T07:40:41.663 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:40:41.663 INFO:teuthology.orchestra.run.smithi131.stdout:Total 13 MB/s | 3.1 MB 00:00 2024-02-05T07:40:41.664 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:40:41.669 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:40:41.669 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:40:41.725 INFO:teuthology.orchestra.run.smithi157.stdout:runc-1.1.12-1.el9.x86_64.rpm 12 MB/s | 3.1 MB 00:00 2024-02-05T07:40:41.726 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:40:41.726 INFO:teuthology.orchestra.run.smithi157.stdout:Total 11 MB/s | 3.1 MB 00:00 2024-02-05T07:40:41.726 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:40:41.732 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:40:41.732 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:40:41.739 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:40:41.739 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:40:41.802 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:40:41.802 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:40:41.933 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:40:42.000 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:40:42.024 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T07:40:42.094 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T07:40:42.539 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T07:40:42.642 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T07:40:42.863 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T07:40:42.863 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:40:42.863 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:40:42.863 INFO:teuthology.orchestra.run.smithi131.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-05T07:40:42.863 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:40:42.863 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:40:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T07:40:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:40:42.955 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:40:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-05T07:40:42.955 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:40:42.955 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:40:43.093 DEBUG:teuthology.parallel:result is None 2024-02-05T07:40:43.212 DEBUG:teuthology.parallel:result is None 2024-02-05T07:40:43.212 INFO:teuthology.run_tasks:Running task install... 2024-02-05T07:40:43.221 DEBUG:teuthology.task.install:project ceph 2024-02-05T07:40:43.221 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '164d3be4a527be775f4d3c343ea591122f1b8921'}, 'extra_packages': ['cephadm']} 2024-02-05T07:40:43.221 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '164d3be4a527be775f4d3c343ea591122f1b8921'} 2024-02-05T07:40:43.221 INFO:teuthology.task.install:Using flavor: default 2024-02-05T07:40:43.227 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-05T07:40:43.227 INFO:teuthology.task.install:extra packages: [] 2024-02-05T07:40:43.227 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '164d3be4a527be775f4d3c343ea591122f1b8921', 'tag': None, 'wait_for_package': False} 2024-02-05T07:40:43.227 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:40:43.230 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '164d3be4a527be775f4d3c343ea591122f1b8921', 'tag': None, 'wait_for_package': False} 2024-02-05T07:40:43.230 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:40:43.373 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/ 2024-02-05T07:40:43.373 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1241.g164d3be4 2024-02-05T07:40:43.446 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/ 2024-02-05T07:40:43.447 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1241.g164d3be4 2024-02-05T07:40:43.546 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-05T07:40:43.546 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-02-05T07:40:43.546 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-05T07:40:43.555 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-05T07:40:43.555 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:40:43.555 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-05T07:40:43.574 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-05T07:40:43.574 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/164d3be4a527be775f4d3c343ea591122f1b8921/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-05T07:40:43.582 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-05T07:40:43.583 DEBUG:teuthology.orchestra.run.smithi131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/164d3be4a527be775f4d3c343ea591122f1b8921/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-05T07:40:43.641 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-05T07:40:43.651 DEBUG:teuthology.orchestra.run.smithi131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-05T07:40:43.721 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-05T07:40:43.737 DEBUG:teuthology.orchestra.run.smithi131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-05T07:40:43.764 INFO:teuthology.orchestra.run.smithi131.stdout:check_obsoletes = 1 2024-02-05T07:40:43.765 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum clean all 2024-02-05T07:40:43.788 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2024-02-05T07:40:43.789 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2024-02-05T07:40:44.004 INFO:teuthology.orchestra.run.smithi131.stdout:74 files removed 2024-02-05T07:40:44.031 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-radosgw 2024-02-05T07:40:44.057 INFO:teuthology.orchestra.run.smithi157.stdout:74 files removed 2024-02-05T07:40:44.084 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw 2024-02-05T07:40:44.669 INFO:teuthology.orchestra.run.smithi131.stdout:ceph packages for x86_64 251 kB/s | 80 kB 00:00 2024-02-05T07:40:44.831 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 187 kB/s | 80 kB 00:00 2024-02-05T07:40:45.069 INFO:teuthology.orchestra.run.smithi131.stdout:ceph noarch packages 29 kB/s | 10 kB 00:00 2024-02-05T07:40:45.257 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-02-05T07:40:45.620 INFO:teuthology.orchestra.run.smithi131.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2024-02-05T07:40:45.958 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 2.9 kB/s | 1.9 kB 00:00 2024-02-05T07:40:46.072 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS-9 - AppStream 16 MB/s | 6.6 MB 00:00 2024-02-05T07:40:46.402 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - AppStream 16 MB/s | 6.6 MB 00:00 2024-02-05T07:40:47.581 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.0 MB 00:00 2024-02-05T07:40:48.069 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2024-02-05T07:40:48.246 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 23 MB/s | 8.0 MB 00:00 2024-02-05T07:40:49.162 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-02-05T07:40:51.467 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 8.7 MB/s | 19 MB 00:02 2024-02-05T07:40:55.840 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 5.9 MB 00:00 2024-02-05T07:40:57.457 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 15 kB 00:00 2024-02-05T07:40:58.267 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 30 MB/s | 20 MB 00:00 2024-02-05T07:41:03.109 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-02-05T07:41:05.211 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-02-05T07:41:05.670 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:41:05.672 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:41:05.672 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-02-05T07:41:05.672 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:41:05.673 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:41:05.673 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 9.7 M 2024-02-05T07:41:05.673 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrading: 2024-02-05T07:41:05.673 INFO:teuthology.orchestra.run.smithi131.stdout: librados2 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 3.4 M 2024-02-05T07:41:05.673 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 3.1 M 2024-02-05T07:41:05.673 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-02-05T07:41:05.673 INFO:teuthology.orchestra.run.smithi131.stdout: boost-program-options 2024-02-05T07:41:05.673 INFO:teuthology.orchestra.run.smithi131.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-05T07:41:05.673 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 5.1 M 2024-02-05T07:41:05.673 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 20 M 2024-02-05T07:41:05.673 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 25 k 2024-02-05T07:41:05.673 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-05T07:41:05.673 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-05T07:41:05.673 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-05T07:41:05.673 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 743 k 2024-02-05T07:41:05.673 INFO:teuthology.orchestra.run.smithi131.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-05T07:41:05.674 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-05T07:41:05.674 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-05T07:41:05.674 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 498 k 2024-02-05T07:41:05.674 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-05T07:41:05.674 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 5.1 M 2024-02-05T07:41:05.674 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-05T07:41:05.674 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-05T07:41:05.674 INFO:teuthology.orchestra.run.smithi131.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-05T07:41:05.674 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse 2024-02-05T07:41:05.674 INFO:teuthology.orchestra.run.smithi131.stdout: x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 45 k 2024-02-05T07:41:05.674 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 131 k 2024-02-05T07:41:05.674 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 162 k 2024-02-05T07:41:05.674 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-05T07:41:05.674 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 320 k 2024-02-05T07:41:05.675 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 298 k 2024-02-05T07:41:05.675 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 99 k 2024-02-05T07:41:05.675 INFO:teuthology.orchestra.run.smithi131.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-05T07:41:05.675 INFO:teuthology.orchestra.run.smithi131.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-05T07:41:05.675 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:41:05.675 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:41:05.675 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:41:05.675 INFO:teuthology.orchestra.run.smithi131.stdout:Install 27 Packages 2024-02-05T07:41:05.675 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrade 2 Packages 2024-02-05T07:41:05.675 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:41:05.675 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 57 M 2024-02-05T07:41:05.676 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:41:06.693 INFO:teuthology.orchestra.run.smithi131.stdout:(1/29): ceph-base-19.0.0-1241.g164d3be4.el9.x86 8.4 MB/s | 5.1 MB 00:00 2024-02-05T07:41:06.751 INFO:teuthology.orchestra.run.smithi131.stdout:(2/29): ceph-selinux-19.0.0-1241.g164d3be4.el9. 433 kB/s | 25 kB 00:00 2024-02-05T07:41:06.860 INFO:teuthology.orchestra.run.smithi131.stdout:(3/29): libcephfs2-19.0.0-1241.g164d3be4.el9.x8 6.7 MB/s | 743 kB 00:00 2024-02-05T07:41:06.944 INFO:teuthology.orchestra.run.smithi131.stdout:(4/29): libradosstriper1-19.0.0-1241.g164d3be4. 5.8 MB/s | 498 kB 00:00 2024-02-05T07:41:07.094 INFO:teuthology.orchestra.run.smithi131.stdout:(5/29): ceph-radosgw-19.0.0-1241.g164d3be4.el9. 9.6 MB/s | 9.7 MB 00:01 2024-02-05T07:41:07.178 INFO:teuthology.orchestra.run.smithi131.stdout:(6/29): python3-ceph-argparse-19.0.0-1241.g164d 542 kB/s | 45 kB 00:00 2024-02-05T07:41:07.286 INFO:teuthology.orchestra.run.smithi131.stdout:(7/29): librgw2-19.0.0-1241.g164d3be4.el9.x86_6 15 MB/s | 5.1 MB 00:00 2024-02-05T07:41:07.312 INFO:teuthology.orchestra.run.smithi131.stdout:(8/29): python3-ceph-common-19.0.0-1241.g164d3b 983 kB/s | 131 kB 00:00 2024-02-05T07:41:07.337 INFO:teuthology.orchestra.run.smithi131.stdout:(9/29): python3-cephfs-19.0.0-1241.g164d3be4.el 3.2 MB/s | 162 kB 00:00 2024-02-05T07:41:07.362 INFO:teuthology.orchestra.run.smithi131.stdout:(10/29): python3-rados-19.0.0-1241.g164d3be4.el 6.2 MB/s | 320 kB 00:00 2024-02-05T07:41:07.387 INFO:teuthology.orchestra.run.smithi131.stdout:(11/29): python3-rbd-19.0.0-1241.g164d3be4.el9. 5.8 MB/s | 298 kB 00:00 2024-02-05T07:41:07.412 INFO:teuthology.orchestra.run.smithi131.stdout:(12/29): python3-rgw-19.0.0-1241.g164d3be4.el9. 1.9 MB/s | 99 kB 00:00 2024-02-05T07:41:07.555 INFO:teuthology.orchestra.run.smithi131.stdout:(13/29): boost-program-options-1.75.0-8.el9.x86 634 kB/s | 106 kB 00:00 2024-02-05T07:41:07.580 INFO:teuthology.orchestra.run.smithi131.stdout:(14/29): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 159 kB 00:00 2024-02-05T07:41:07.605 INFO:teuthology.orchestra.run.smithi131.stdout:(15/29): librabbitmq-0.11.0-7.el9.x86_64.rpm 882 kB/s | 44 kB 00:00 2024-02-05T07:41:07.672 INFO:teuthology.orchestra.run.smithi131.stdout:(16/29): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 661 kB 00:00 2024-02-05T07:41:07.697 INFO:teuthology.orchestra.run.smithi131.stdout:(17/29): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2024-02-05T07:41:07.714 INFO:teuthology.orchestra.run.smithi131.stdout:(18/29): python3-prettytable-0.7.2-27.el9.noarc 994 kB/s | 41 kB 00:00 2024-02-05T07:41:07.932 INFO:teuthology.orchestra.run.smithi131.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 500 for https://mirror.servaxnet.com/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 162.255.226.75) 2024-02-05T07:41:08.514 INFO:teuthology.orchestra.run.smithi131.stdout:(19/29): gperftools-libs-2.9.1-2.el9.x86_64.rpm 378 kB/s | 309 kB 00:00 2024-02-05T07:41:08.556 INFO:teuthology.orchestra.run.smithi131.stdout:(20/29): libarrow-doc-9.0.0-10.el9.noarch.rpm 624 kB/s | 26 kB 00:00 2024-02-05T07:41:08.598 INFO:teuthology.orchestra.run.smithi131.stdout:(21/29): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-02-05T07:41:08.665 INFO:teuthology.orchestra.run.smithi131.stdout:(22/29): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-02-05T07:41:09.266 INFO:teuthology.orchestra.run.smithi131.stdout:(23/29): ceph-common-19.0.0-1241.g164d3be4.el9. 6.2 MB/s | 20 MB 00:03 2024-02-05T07:41:10.561 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 5.9 MB 00:01 2024-02-05T07:41:11.887 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 82 kB/s | 15 kB 00:00 2024-02-05T07:41:12.957 INFO:teuthology.orchestra.run.smithi131.stdout:(24/29): re2-20211101-3.el9.x86_64.rpm 52 kB/s | 192 kB 00:03 2024-02-05T07:41:13.065 INFO:teuthology.orchestra.run.smithi131.stdout:(25/29): parquet-libs-9.0.0-10.el9.x86_64.rpm 190 kB/s | 838 kB 00:04 2024-02-05T07:41:13.332 INFO:teuthology.orchestra.run.smithi131.stdout:(26/29): librados2-19.0.0-1241.g164d3be4.el9.x8 13 MB/s | 3.4 MB 00:00 2024-02-05T07:41:13.508 INFO:teuthology.orchestra.run.smithi131.stdout:(27/29): librbd1-19.0.0-1241.g164d3be4.el9.x86_ 18 MB/s | 3.1 MB 00:00 2024-02-05T07:41:16.513 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 4.5 MB/s | 20 MB 00:04 2024-02-05T07:41:18.465 INFO:teuthology.orchestra.run.smithi131.stdout:(28/29): thrift-0.15.0-2.el9.x86_64.rpm 294 kB/s | 1.6 MB 00:05 2024-02-05T07:41:21.329 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-02-05T07:41:23.876 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:41:23.878 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:41:23.878 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-02-05T07:41:23.878 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:41:23.878 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:41:23.878 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 9.7 M 2024-02-05T07:41:23.878 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-02-05T07:41:23.878 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 3.4 M 2024-02-05T07:41:23.878 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 3.1 M 2024-02-05T07:41:23.878 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-02-05T07:41:23.878 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options 2024-02-05T07:41:23.878 INFO:teuthology.orchestra.run.smithi157.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-05T07:41:23.879 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 5.1 M 2024-02-05T07:41:23.879 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 20 M 2024-02-05T07:41:23.879 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 25 k 2024-02-05T07:41:23.879 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-05T07:41:23.879 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-05T07:41:23.879 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-05T07:41:23.879 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 743 k 2024-02-05T07:41:23.879 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-05T07:41:23.879 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-05T07:41:23.879 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-05T07:41:23.879 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 498 k 2024-02-05T07:41:23.879 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-05T07:41:23.879 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 5.1 M 2024-02-05T07:41:23.879 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-05T07:41:23.880 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-05T07:41:23.880 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-05T07:41:23.880 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse 2024-02-05T07:41:23.880 INFO:teuthology.orchestra.run.smithi157.stdout: x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 45 k 2024-02-05T07:41:23.880 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 131 k 2024-02-05T07:41:23.880 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 162 k 2024-02-05T07:41:23.880 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-05T07:41:23.880 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 320 k 2024-02-05T07:41:23.880 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 298 k 2024-02-05T07:41:23.880 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 99 k 2024-02-05T07:41:23.880 INFO:teuthology.orchestra.run.smithi157.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-05T07:41:23.880 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-05T07:41:23.880 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:41:23.880 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:41:23.881 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:41:23.881 INFO:teuthology.orchestra.run.smithi157.stdout:Install 27 Packages 2024-02-05T07:41:23.881 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 2 Packages 2024-02-05T07:41:23.881 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:41:23.881 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 57 M 2024-02-05T07:41:23.881 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:41:24.972 INFO:teuthology.orchestra.run.smithi157.stdout:(1/29): ceph-base-19.0.0-1241.g164d3be4.el9.x86 7.0 MB/s | 5.1 MB 00:00 2024-02-05T07:41:25.039 INFO:teuthology.orchestra.run.smithi157.stdout:(2/29): ceph-selinux-19.0.0-1241.g164d3be4.el9. 378 kB/s | 25 kB 00:00 2024-02-05T07:41:25.190 INFO:teuthology.orchestra.run.smithi157.stdout:(3/29): libcephfs2-19.0.0-1241.g164d3be4.el9.x8 4.8 MB/s | 743 kB 00:00 2024-02-05T07:41:25.290 INFO:teuthology.orchestra.run.smithi157.stdout:(4/29): libradosstriper1-19.0.0-1241.g164d3be4. 4.9 MB/s | 498 kB 00:00 2024-02-05T07:41:25.949 INFO:teuthology.orchestra.run.smithi157.stdout:(5/29): ceph-common-19.0.0-1241.g164d3be4.el9.x 12 MB/s | 20 MB 00:01 2024-02-05T07:41:26.024 INFO:teuthology.orchestra.run.smithi157.stdout:(6/29): librgw2-19.0.0-1241.g164d3be4.el9.x86_6 6.9 MB/s | 5.1 MB 00:00 2024-02-05T07:41:26.049 INFO:teuthology.orchestra.run.smithi157.stdout:(7/29): python3-ceph-argparse-19.0.0-1241.g164d 451 kB/s | 45 kB 00:00 2024-02-05T07:41:26.074 INFO:teuthology.orchestra.run.smithi157.stdout:(8/29): python3-ceph-common-19.0.0-1241.g164d3b 2.6 MB/s | 131 kB 00:00 2024-02-05T07:41:26.099 INFO:teuthology.orchestra.run.smithi157.stdout:(9/29): python3-cephfs-19.0.0-1241.g164d3be4.el 3.2 MB/s | 162 kB 00:00 2024-02-05T07:41:26.150 INFO:teuthology.orchestra.run.smithi157.stdout:(10/29): python3-rados-19.0.0-1241.g164d3be4.el 4.2 MB/s | 320 kB 00:00 2024-02-05T07:41:26.183 INFO:teuthology.orchestra.run.smithi157.stdout:(11/29): python3-rbd-19.0.0-1241.g164d3be4.el9. 3.5 MB/s | 298 kB 00:00 2024-02-05T07:41:26.208 INFO:teuthology.orchestra.run.smithi157.stdout:(12/29): python3-rgw-19.0.0-1241.g164d3be4.el9. 1.7 MB/s | 99 kB 00:00 2024-02-05T07:41:26.367 INFO:teuthology.orchestra.run.smithi157.stdout:(13/29): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 159 kB 00:00 2024-02-05T07:41:26.392 INFO:teuthology.orchestra.run.smithi157.stdout:(14/29): boost-program-options-1.75.0-8.el9.x86 507 kB/s | 106 kB 00:00 2024-02-05T07:41:26.418 INFO:teuthology.orchestra.run.smithi157.stdout:(15/29): librabbitmq-0.11.0-7.el9.x86_64.rpm 880 kB/s | 44 kB 00:00 2024-02-05T07:41:26.485 INFO:teuthology.orchestra.run.smithi157.stdout:(16/29): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 661 kB 00:00 2024-02-05T07:41:26.510 INFO:teuthology.orchestra.run.smithi157.stdout:(17/29): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2024-02-05T07:41:26.535 INFO:teuthology.orchestra.run.smithi157.stdout:(18/29): python3-prettytable-0.7.2-27.el9.noarc 830 kB/s | 41 kB 00:00 2024-02-05T07:41:26.681 INFO:teuthology.orchestra.run.smithi131.stdout:(29/29): libarrow-9.0.0-10.el9.x86_64.rpm 238 kB/s | 4.4 MB 00:18 2024-02-05T07:41:26.688 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:41:26.688 INFO:teuthology.orchestra.run.smithi131.stdout:Total 2.7 MB/s | 57 MB 00:21 2024-02-05T07:41:26.688 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:41:26.724 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:41:26.724 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:41:26.785 INFO:teuthology.orchestra.run.smithi157.stdout:(19/29): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.1 MB/s | 309 kB 00:00 2024-02-05T07:41:26.852 INFO:teuthology.orchestra.run.smithi157.stdout:(20/29): libarrow-doc-9.0.0-10.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-02-05T07:41:26.919 INFO:teuthology.orchestra.run.smithi157.stdout:(21/29): liboath-2.6.7-2.el9.x86_64.rpm 729 kB/s | 49 kB 00:00 2024-02-05T07:41:27.003 INFO:teuthology.orchestra.run.smithi157.stdout:(22/29): libunwind-1.6.2-1.el9.x86_64.rpm 809 kB/s | 67 kB 00:00 2024-02-05T07:41:27.103 INFO:teuthology.orchestra.run.smithi157.stdout:(23/29): libarrow-9.0.0-10.el9.x86_64.rpm 7.8 MB/s | 4.4 MB 00:00 2024-02-05T07:41:27.170 INFO:teuthology.orchestra.run.smithi157.stdout:(24/29): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-02-05T07:41:27.202 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:41:27.202 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:41:27.204 INFO:teuthology.orchestra.run.smithi157.stdout:(25/29): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.1 MB/s | 838 kB 00:00 2024-02-05T07:41:27.296 INFO:teuthology.orchestra.run.smithi157.stdout:(26/29): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-02-05T07:41:27.546 INFO:teuthology.orchestra.run.smithi157.stdout:(27/29): librados2-19.0.0-1241.g164d3be4.el9.x8 9.8 MB/s | 3.4 MB 00:00 2024-02-05T07:41:27.604 INFO:teuthology.orchestra.run.smithi157.stdout:(28/29): librbd1-19.0.0-1241.g164d3be4.el9.x86_ 10 MB/s | 3.1 MB 00:00 2024-02-05T07:41:28.533 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:41:28.616 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/31 2024-02-05T07:41:28.644 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/31 2024-02-05T07:41:28.719 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/31 2024-02-05T07:41:29.044 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/31 2024-02-05T07:41:29.110 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 5/31 2024-02-05T07:41:29.200 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 5/31 2024-02-05T07:41:29.288 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rados-2:19.0.0-1241.g164d3be4.el9.x86_64 6/31 2024-02-05T07:41:29.320 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 7/31 2024-02-05T07:41:29.381 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 7/31 2024-02-05T07:41:29.411 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/31 2024-02-05T07:41:29.646 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-argparse-2:19.0.0-1241.g164d3be4.el9. 9/31 2024-02-05T07:41:29.714 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cephfs-2:19.0.0-1241.g164d3be4.el9.x86_64 10/31 2024-02-05T07:41:29.739 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libradosstriper1-2:19.0.0-1241.g164d3be4.el9.x86_6 11/31 2024-02-05T07:41:29.796 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1241.g164d3be4.el9.x86_6 11/31 2024-02-05T07:41:29.902 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/31 2024-02-05T07:41:29.963 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/31 2024-02-05T07:41:30.003 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/31 2024-02-05T07:41:30.036 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/31 2024-02-05T07:41:30.064 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/31 2024-02-05T07:41:30.097 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 17/31 2024-02-05T07:41:30.409 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 18/31 2024-02-05T07:41:30.435 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 19/31 2024-02-05T07:41:30.508 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 19/31 2024-02-05T07:41:30.548 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rbd-2:19.0.0-1241.g164d3be4.el9.x86_64 20/31 2024-02-05T07:41:30.594 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 21/31 2024-02-05T07:41:31.048 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 22/31 2024-02-05T07:41:31.079 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librgw2-2:19.0.0-1241.g164d3be4.el9.x86_64 23/31 2024-02-05T07:41:31.136 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librgw2-2:19.0.0-1241.g164d3be4.el9.x86_64 23/31 2024-02-05T07:41:31.202 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rgw-2:19.0.0-1241.g164d3be4.el9.x86_64 24/31 2024-02-05T07:41:31.231 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-common-2:19.0.0-1241.g164d3be4.el9.x8 25/31 2024-02-05T07:41:32.196 INFO:teuthology.orchestra.run.smithi157.stdout:(29/29): ceph-radosgw-19.0.0-1241.g164d3be4.el9 1.2 MB/s | 9.7 MB 00:07 2024-02-05T07:41:32.200 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:41:32.200 INFO:teuthology.orchestra.run.smithi157.stdout:Total 6.9 MB/s | 57 MB 00:08 2024-02-05T07:41:32.201 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:41:32.236 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:41:32.236 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:41:32.724 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:41:32.724 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:41:33.402 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 26/31 2024-02-05T07:41:33.444 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 26/31 2024-02-05T07:41:33.972 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 26/31 2024-02-05T07:41:34.014 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 27/31 2024-02-05T07:41:34.062 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:41:34.088 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 27/31 2024-02-05T07:41:34.088 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-05T07:41:34.088 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-05T07:41:34.088 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:41:34.132 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-selinux-2:19.0.0-1241.g164d3be4.el9.x86_64 28/31 2024-02-05T07:41:34.146 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/31 2024-02-05T07:41:34.183 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/31 2024-02-05T07:41:34.234 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/31 2024-02-05T07:41:34.560 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/31 2024-02-05T07:41:34.633 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 5/31 2024-02-05T07:41:34.707 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 5/31 2024-02-05T07:41:34.795 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:19.0.0-1241.g164d3be4.el9.x86_64 6/31 2024-02-05T07:41:34.818 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 7/31 2024-02-05T07:41:34.879 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 7/31 2024-02-05T07:41:34.909 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/31 2024-02-05T07:41:34.944 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:19.0.0-1241.g164d3be4.el9. 9/31 2024-02-05T07:41:35.012 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:19.0.0-1241.g164d3be4.el9.x86_64 10/31 2024-02-05T07:41:35.037 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:19.0.0-1241.g164d3be4.el9.x86_6 11/31 2024-02-05T07:41:35.085 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1241.g164d3be4.el9.x86_6 11/31 2024-02-05T07:41:35.193 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/31 2024-02-05T07:41:35.228 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/31 2024-02-05T07:41:35.269 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/31 2024-02-05T07:41:35.309 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/31 2024-02-05T07:41:35.338 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/31 2024-02-05T07:41:35.370 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 17/31 2024-02-05T07:41:35.696 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 18/31 2024-02-05T07:41:35.725 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 19/31 2024-02-05T07:41:35.798 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 19/31 2024-02-05T07:41:35.830 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:19.0.0-1241.g164d3be4.el9.x86_64 20/31 2024-02-05T07:41:35.872 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 21/31 2024-02-05T07:41:36.321 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 22/31 2024-02-05T07:41:36.344 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:19.0.0-1241.g164d3be4.el9.x86_64 23/31 2024-02-05T07:41:36.399 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:19.0.0-1241.g164d3be4.el9.x86_64 23/31 2024-02-05T07:41:36.475 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:19.0.0-1241.g164d3be4.el9.x86_64 24/31 2024-02-05T07:41:36.505 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:19.0.0-1241.g164d3be4.el9.x8 25/31 2024-02-05T07:41:38.708 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 26/31 2024-02-05T07:41:38.744 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 26/31 2024-02-05T07:41:39.280 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 26/31 2024-02-05T07:41:39.322 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 27/31 2024-02-05T07:41:39.414 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 27/31 2024-02-05T07:41:39.414 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-05T07:41:39.414 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-05T07:41:39.414 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:41:39.440 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:19.0.0-1241.g164d3be4.el9.x86_64 28/31 2024-02-05T07:41:43.305 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1241.g164d3be4.el9.x86_64 28/31 2024-02-05T07:41:43.305 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /sys 2024-02-05T07:41:43.305 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /proc 2024-02-05T07:41:43.305 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /mnt 2024-02-05T07:41:43.305 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /var/tmp 2024-02-05T07:41:43.305 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /home 2024-02-05T07:41:43.305 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /root 2024-02-05T07:41:43.306 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /tmp 2024-02-05T07:41:43.306 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:41:44.222 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-radosgw-2:19.0.0-1241.g164d3be4.el9.x86_64 29/31 2024-02-05T07:41:44.253 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1241.g164d3be4.el9.x86_64 29/31 2024-02-05T07:41:44.253 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:41:44.253 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-05T07:41:44.253 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-05T07:41:44.253 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-05T07:41:44.253 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:41:44.255 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 30/31 2024-02-05T07:41:44.342 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 30/31 2024-02-05T07:41:44.342 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 31/31 2024-02-05T07:41:45.449 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 31/31 2024-02-05T07:41:45.449 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 1/31 2024-02-05T07:41:45.449 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 2/31 2024-02-05T07:41:45.449 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-radosgw-2:19.0.0-1241.g164d3be4.el9.x86_64 3/31 2024-02-05T07:41:45.449 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-selinux-2:19.0.0-1241.g164d3be4.el9.x86_64 4/31 2024-02-05T07:41:45.449 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 5/31 2024-02-05T07:41:45.449 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libradosstriper1-2:19.0.0-1241.g164d3be4.el9.x86_6 6/31 2024-02-05T07:41:45.449 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librgw2-2:19.0.0-1241.g164d3be4.el9.x86_64 7/31 2024-02-05T07:41:45.449 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1241.g164d3be4.el9. 8/31 2024-02-05T07:41:45.450 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-common-2:19.0.0-1241.g164d3be4.el9.x8 9/31 2024-02-05T07:41:45.450 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cephfs-2:19.0.0-1241.g164d3be4.el9.x86_64 10/31 2024-02-05T07:41:45.450 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rados-2:19.0.0-1241.g164d3be4.el9.x86_64 11/31 2024-02-05T07:41:45.450 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rbd-2:19.0.0-1241.g164d3be4.el9.x86_64 12/31 2024-02-05T07:41:45.450 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rgw-2:19.0.0-1241.g164d3be4.el9.x86_64 13/31 2024-02-05T07:41:45.450 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/31 2024-02-05T07:41:45.450 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/31 2024-02-05T07:41:45.450 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/31 2024-02-05T07:41:45.450 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/31 2024-02-05T07:41:45.450 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/31 2024-02-05T07:41:45.450 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/31 2024-02-05T07:41:45.450 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 20/31 2024-02-05T07:41:45.450 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 21/31 2024-02-05T07:41:45.450 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 22/31 2024-02-05T07:41:45.450 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 23/31 2024-02-05T07:41:45.451 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 24/31 2024-02-05T07:41:45.451 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 25/31 2024-02-05T07:41:45.451 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : re2-1:20211101-3.el9.x86_64 26/31 2024-02-05T07:41:45.451 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 27/31 2024-02-05T07:41:45.451 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 28/31 2024-02-05T07:41:45.451 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 29/31 2024-02-05T07:41:45.451 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 30/31 2024-02-05T07:41:46.389 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 31/31 2024-02-05T07:41:46.389 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:41:46.389 INFO:teuthology.orchestra.run.smithi131.stdout:Upgraded: 2024-02-05T07:41:46.389 INFO:teuthology.orchestra.run.smithi131.stdout: librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:46.389 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:46.389 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:41:46.389 INFO:teuthology.orchestra.run.smithi131.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-05T07:41:46.389 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:46.389 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:46.389 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:46.389 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:46.389 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-05T07:41:46.389 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-05T07:41:46.389 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-05T07:41:46.390 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:46.390 INFO:teuthology.orchestra.run.smithi131.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-05T07:41:46.390 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-05T07:41:46.390 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-05T07:41:46.390 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:46.390 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-05T07:41:46.390 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:46.390 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-05T07:41:46.390 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-05T07:41:46.390 INFO:teuthology.orchestra.run.smithi131.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-05T07:41:46.390 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:46.390 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:46.390 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:46.390 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-05T07:41:46.390 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:46.391 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:46.391 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:46.391 INFO:teuthology.orchestra.run.smithi131.stdout: re2-1:20211101-3.el9.x86_64 2024-02-05T07:41:46.391 INFO:teuthology.orchestra.run.smithi131.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-05T07:41:46.391 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:41:46.391 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:41:46.720 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-test 2024-02-05T07:41:47.295 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:44 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:41:47.460 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:41:47.461 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:41:47.461 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-02-05T07:41:47.461 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:41:47.461 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:41:47.461 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 47 M 2024-02-05T07:41:47.461 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-02-05T07:41:47.461 INFO:teuthology.orchestra.run.smithi131.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-05T07:41:47.461 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 163 k 2024-02-05T07:41:47.461 INFO:teuthology.orchestra.run.smithi131.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-05T07:41:47.462 INFO:teuthology.orchestra.run.smithi131.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-05T07:41:47.462 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-05T07:41:47.462 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:41:47.462 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:41:47.462 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:41:47.462 INFO:teuthology.orchestra.run.smithi131.stdout:Install 6 Packages 2024-02-05T07:41:47.462 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:41:47.462 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 48 M 2024-02-05T07:41:47.463 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 204 M 2024-02-05T07:41:47.463 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:41:47.673 INFO:teuthology.orchestra.run.smithi131.stdout:(1/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2024-02-05T07:41:47.715 INFO:teuthology.orchestra.run.smithi131.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-02-05T07:41:47.765 INFO:teuthology.orchestra.run.smithi131.stdout:(3/6): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-02-05T07:41:47.816 INFO:teuthology.orchestra.run.smithi131.stdout:(4/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-02-05T07:41:47.849 INFO:teuthology.orchestra.run.smithi131.stdout:(5/6): libcephsqlite-19.0.0-1241.g164d3be4.el9. 423 kB/s | 163 kB 00:00 2024-02-05T07:41:48.953 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1241.g164d3be4.el9.x86_64 28/31 2024-02-05T07:41:48.953 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2024-02-05T07:41:48.953 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2024-02-05T07:41:48.954 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2024-02-05T07:41:48.954 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2024-02-05T07:41:48.954 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2024-02-05T07:41:48.954 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /root 2024-02-05T07:41:48.954 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2024-02-05T07:41:48.954 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:41:49.758 INFO:teuthology.orchestra.run.smithi131.stdout:(6/6): ceph-test-19.0.0-1241.g164d3be4.el9.x86_ 21 MB/s | 47 MB 00:02 2024-02-05T07:41:49.760 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:41:49.761 INFO:teuthology.orchestra.run.smithi131.stdout:Total 21 MB/s | 48 MB 00:02 2024-02-05T07:41:49.761 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:41:49.781 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:41:49.781 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:41:49.880 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:19.0.0-1241.g164d3be4.el9.x86_64 29/31 2024-02-05T07:41:49.904 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1241.g164d3be4.el9.x86_64 29/31 2024-02-05T07:41:49.904 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:41:49.904 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-05T07:41:49.904 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-05T07:41:49.905 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-05T07:41:49.905 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:41:49.906 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 30/31 2024-02-05T07:41:49.983 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 30/31 2024-02-05T07:41:49.983 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 31/31 2024-02-05T07:41:50.161 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:41:50.162 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:41:50.733 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:41:50.797 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-05T07:41:50.840 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-05T07:41:50.883 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-05T07:41:50.931 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-05T07:41:50.963 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephsqlite-2:19.0.0-1241.g164d3be4.el9.x86_64 5/6 2024-02-05T07:41:51.119 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 31/31 2024-02-05T07:41:51.119 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 1/31 2024-02-05T07:41:51.119 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 2/31 2024-02-05T07:41:51.119 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:19.0.0-1241.g164d3be4.el9.x86_64 3/31 2024-02-05T07:41:51.119 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:19.0.0-1241.g164d3be4.el9.x86_64 4/31 2024-02-05T07:41:51.120 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 5/31 2024-02-05T07:41:51.120 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:19.0.0-1241.g164d3be4.el9.x86_6 6/31 2024-02-05T07:41:51.120 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:19.0.0-1241.g164d3be4.el9.x86_64 7/31 2024-02-05T07:41:51.120 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1241.g164d3be4.el9. 8/31 2024-02-05T07:41:51.120 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:19.0.0-1241.g164d3be4.el9.x8 9/31 2024-02-05T07:41:51.120 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:19.0.0-1241.g164d3be4.el9.x86_64 10/31 2024-02-05T07:41:51.120 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:19.0.0-1241.g164d3be4.el9.x86_64 11/31 2024-02-05T07:41:51.120 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:19.0.0-1241.g164d3be4.el9.x86_64 12/31 2024-02-05T07:41:51.120 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:19.0.0-1241.g164d3be4.el9.x86_64 13/31 2024-02-05T07:41:51.120 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/31 2024-02-05T07:41:51.120 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/31 2024-02-05T07:41:51.120 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/31 2024-02-05T07:41:51.120 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/31 2024-02-05T07:41:51.121 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/31 2024-02-05T07:41:51.121 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/31 2024-02-05T07:41:51.121 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 20/31 2024-02-05T07:41:51.121 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 21/31 2024-02-05T07:41:51.121 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 22/31 2024-02-05T07:41:51.121 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 23/31 2024-02-05T07:41:51.121 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 24/31 2024-02-05T07:41:51.121 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 25/31 2024-02-05T07:41:51.121 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : re2-1:20211101-3.el9.x86_64 26/31 2024-02-05T07:41:51.121 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 27/31 2024-02-05T07:41:51.122 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 28/31 2024-02-05T07:41:51.122 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 29/31 2024-02-05T07:41:51.122 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 30/31 2024-02-05T07:41:52.256 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 31/31 2024-02-05T07:41:52.256 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:41:52.256 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-02-05T07:41:52.256 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:52.256 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:52.256 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:41:52.256 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-05T07:41:52.256 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:52.257 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:52.257 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:52.257 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:52.257 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-05T07:41:52.257 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-05T07:41:52.257 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-05T07:41:52.257 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:52.257 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-05T07:41:52.257 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-05T07:41:52.257 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-05T07:41:52.257 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:52.257 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-05T07:41:52.257 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:52.258 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-05T07:41:52.258 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-05T07:41:52.258 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-05T07:41:52.258 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:52.258 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:52.258 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:52.258 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-05T07:41:52.258 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:52.258 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:52.258 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:52.258 INFO:teuthology.orchestra.run.smithi157.stdout: re2-1:20211101-3.el9.x86_64 2024-02-05T07:41:52.258 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-05T07:41:52.258 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:41:52.258 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:41:52.569 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-test 2024-02-05T07:41:53.131 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:32 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:41:53.286 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:41:53.287 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:41:53.287 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-02-05T07:41:53.287 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:41:53.288 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:41:53.288 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 47 M 2024-02-05T07:41:53.288 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-02-05T07:41:53.288 INFO:teuthology.orchestra.run.smithi157.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-05T07:41:53.288 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 163 k 2024-02-05T07:41:53.288 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-05T07:41:53.288 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-05T07:41:53.288 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-05T07:41:53.288 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:41:53.288 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:41:53.288 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:41:53.288 INFO:teuthology.orchestra.run.smithi157.stdout:Install 6 Packages 2024-02-05T07:41:53.288 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:41:53.289 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 48 M 2024-02-05T07:41:53.289 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 204 M 2024-02-05T07:41:53.289 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:41:53.486 INFO:teuthology.orchestra.run.smithi157.stdout:(1/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 300 kB 00:00 2024-02-05T07:41:53.528 INFO:teuthology.orchestra.run.smithi157.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-02-05T07:41:53.586 INFO:teuthology.orchestra.run.smithi157.stdout:(3/6): jq-1.6-16.el9.x86_64.rpm 3.1 MB/s | 186 kB 00:00 2024-02-05T07:41:53.645 INFO:teuthology.orchestra.run.smithi157.stdout:(4/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-02-05T07:41:53.670 INFO:teuthology.orchestra.run.smithi157.stdout:(5/6): libcephsqlite-19.0.0-1241.g164d3be4.el9. 429 kB/s | 163 kB 00:00 2024-02-05T07:41:55.401 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1241.g164d3be4.el9.x86_64 5/6 2024-02-05T07:41:55.484 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-test-2:19.0.0-1241.g164d3be4.el9.x86_64 6/6 2024-02-05T07:41:58.075 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-test-2:19.0.0-1241.g164d3be4.el9.x86_64 6/6 2024-02-05T07:41:58.075 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-test-2:19.0.0-1241.g164d3be4.el9.x86_64 1/6 2024-02-05T07:41:58.075 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephsqlite-2:19.0.0-1241.g164d3be4.el9.x86_64 2/6 2024-02-05T07:41:58.075 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-05T07:41:58.075 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-05T07:41:58.075 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-05T07:41:58.443 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-05T07:41:58.443 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:41:58.443 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:41:58.443 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:58.443 INFO:teuthology.orchestra.run.smithi131.stdout: jq-1.6-16.el9.x86_64 2024-02-05T07:41:58.443 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:41:58.443 INFO:teuthology.orchestra.run.smithi131.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-05T07:41:58.443 INFO:teuthology.orchestra.run.smithi131.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-05T07:41:58.443 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-05T07:41:58.443 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:41:58.444 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:41:58.712 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph 2024-02-05T07:41:59.277 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:56 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:41:59.437 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:41:59.440 INFO:teuthology.orchestra.run.smithi131.stdout:========================================================================================= 2024-02-05T07:41:59.440 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-02-05T07:41:59.440 INFO:teuthology.orchestra.run.smithi131.stdout:========================================================================================= 2024-02-05T07:41:59.440 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:41:59.440 INFO:teuthology.orchestra.run.smithi131.stdout: ceph x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 6.6 k 2024-02-05T07:41:59.440 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-02-05T07:41:59.440 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 2.1 M 2024-02-05T07:41:59.440 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 1.5 M 2024-02-05T07:41:59.440 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 247 k 2024-02-05T07:41:59.440 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 4.7 M 2024-02-05T07:41:59.440 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 17 M 2024-02-05T07:41:59.441 INFO:teuthology.orchestra.run.smithi131.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-05T07:41:59.441 INFO:teuthology.orchestra.run.smithi131.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-05T07:41:59.441 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-05T07:41:59.441 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-05T07:41:59.441 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-05T07:41:59.441 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-05T07:41:59.441 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-05T07:41:59.441 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-05T07:41:59.441 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-05T07:41:59.441 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-05T07:41:59.441 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-05T07:41:59.441 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-05T07:41:59.441 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-05T07:41:59.441 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-05T07:41:59.441 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-05T07:41:59.442 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-05T07:41:59.442 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-05T07:41:59.442 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-05T07:41:59.442 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-05T07:41:59.442 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-05T07:41:59.442 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-05T07:41:59.442 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-05T07:41:59.442 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-05T07:41:59.442 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-05T07:41:59.442 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-05T07:41:59.442 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-05T07:41:59.442 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-05T07:41:59.442 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-05T07:41:59.442 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-05T07:41:59.442 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-05T07:41:59.442 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:41:59.443 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:41:59.443 INFO:teuthology.orchestra.run.smithi131.stdout:========================================================================================= 2024-02-05T07:41:59.443 INFO:teuthology.orchestra.run.smithi131.stdout:Install 36 Packages 2024-02-05T07:41:59.443 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:41:59.443 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 30 M 2024-02-05T07:41:59.443 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 107 M 2024-02-05T07:41:59.443 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:41:59.889 INFO:teuthology.orchestra.run.smithi131.stdout:(1/36): ceph-19.0.0-1241.g164d3be4.el9.x86_64.r 42 kB/s | 6.6 kB 00:00 2024-02-05T07:42:00.140 INFO:teuthology.orchestra.run.smithi131.stdout:(2/36): ceph-mds-19.0.0-1241.g164d3be4.el9.x86_ 5.2 MB/s | 2.1 MB 00:00 2024-02-05T07:42:00.189 INFO:teuthology.orchestra.run.smithi131.stdout:(3/36): ceph-mgr-19.0.0-1241.g164d3be4.el9.x86_ 3.3 MB/s | 1.5 MB 00:00 2024-02-05T07:42:00.290 INFO:teuthology.orchestra.run.smithi131.stdout:(4/36): ceph-mgr-modules-core-19.0.0-1241.g164d 2.4 MB/s | 247 kB 00:00 2024-02-05T07:42:00.532 INFO:teuthology.orchestra.run.smithi131.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 773 kB/s | 187 kB 00:00 2024-02-05T07:42:00.599 INFO:teuthology.orchestra.run.smithi131.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2024-02-05T07:42:00.649 INFO:teuthology.orchestra.run.smithi131.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 633 kB/s | 32 kB 00:00 2024-02-05T07:42:00.699 INFO:teuthology.orchestra.run.smithi131.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 945 kB/s | 47 kB 00:00 2024-02-05T07:42:00.749 INFO:teuthology.orchestra.run.smithi131.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 772 kB/s | 39 kB 00:00 2024-02-05T07:42:00.817 INFO:teuthology.orchestra.run.smithi131.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-02-05T07:42:00.925 INFO:teuthology.orchestra.run.smithi131.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 11 MB/s | 1.1 MB 00:00 2024-02-05T07:42:01.000 INFO:teuthology.orchestra.run.smithi131.stdout:(12/36): ceph-mon-19.0.0-1241.g164d3be4.el9.x86 4.2 MB/s | 4.7 MB 00:01 2024-02-05T07:42:01.026 INFO:teuthology.orchestra.run.smithi131.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-02-05T07:42:01.051 INFO:teuthology.orchestra.run.smithi131.stdout:(14/36): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-02-05T07:42:01.076 INFO:teuthology.orchestra.run.smithi131.stdout:(15/36): python3-requests-2.25.1-8.el9.noarch.r 2.2 MB/s | 113 kB 00:00 2024-02-05T07:42:01.101 INFO:teuthology.orchestra.run.smithi131.stdout:(16/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-02-05T07:42:01.260 INFO:teuthology.orchestra.run.smithi131.stdout:(17/36): ceph-osd-19.0.0-1241.g164d3be4.el9.x86 15 MB/s | 17 MB 00:01 2024-02-05T07:42:01.385 INFO:teuthology.orchestra.run.smithi131.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 72 kB/s | 22 kB 00:00 2024-02-05T07:42:01.527 INFO:teuthology.orchestra.run.smithi131.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 179 kB/s | 43 kB 00:00 2024-02-05T07:42:01.694 INFO:teuthology.orchestra.run.smithi131.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 255 kB/s | 151 kB 00:00 2024-02-05T07:42:01.737 INFO:teuthology.orchestra.run.smithi131.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 491 kB/s | 172 kB 00:00 2024-02-05T07:42:01.820 INFO:teuthology.orchestra.run.smithi131.stdout:(22/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-02-05T07:42:01.837 INFO:teuthology.orchestra.run.smithi131.stdout:(23/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 75 kB/s | 11 kB 00:00 2024-02-05T07:42:01.854 INFO:teuthology.orchestra.run.smithi131.stdout:(24/36): python3-jaraco-classes-3.2.1-5.el9.noa 151 kB/s | 18 kB 00:00 2024-02-05T07:42:01.871 INFO:teuthology.orchestra.run.smithi131.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-02-05T07:42:01.896 INFO:teuthology.orchestra.run.smithi131.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 467 kB/s | 19 kB 00:00 2024-02-05T07:42:01.913 INFO:teuthology.orchestra.run.smithi131.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 336 kB/s | 20 kB 00:00 2024-02-05T07:42:02.105 INFO:teuthology.orchestra.run.smithi131.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 198 kB/s | 46 kB 00:00 2024-02-05T07:42:02.280 INFO:teuthology.orchestra.run.smithi131.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 205 kB/s | 79 kB 00:00 2024-02-05T07:42:03.506 INFO:teuthology.orchestra.run.smithi131.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 171 kB/s | 272 kB 00:01 2024-02-05T07:42:03.547 INFO:teuthology.orchestra.run.smithi131.stdout:(31/36): python3-portend-3.1.0-2.el9.noarch.rpm 11 kB/s | 16 kB 00:01 2024-02-05T07:42:04.064 INFO:teuthology.orchestra.run.smithi131.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 50 kB/s | 90 kB 00:01 2024-02-05T07:42:04.306 INFO:teuthology.orchestra.run.smithi131.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 45 kB/s | 36 kB 00:00 2024-02-05T07:42:05.238 INFO:teuthology.orchestra.run.smithi131.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 136 kB/s | 230 kB 00:01 2024-02-05T07:42:05.852 INFO:teuthology.orchestra.run.smithi157.stdout:(6/6): ceph-test-19.0.0-1241.g164d3be4.el9.x86_ 3.8 MB/s | 47 MB 00:12 2024-02-05T07:42:05.856 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:42:05.856 INFO:teuthology.orchestra.run.smithi157.stdout:Total 3.8 MB/s | 48 MB 00:12 2024-02-05T07:42:05.856 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:42:05.876 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:42:05.876 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:42:06.258 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:42:06.258 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:42:06.631 INFO:teuthology.orchestra.run.smithi131.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 8.6 kB/s | 20 kB 00:02 2024-02-05T07:42:06.827 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:42:06.881 INFO:teuthology.orchestra.run.smithi131.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 152 kB/s | 427 kB 00:02 2024-02-05T07:42:06.887 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:42:06.887 INFO:teuthology.orchestra.run.smithi131.stdout:Total 4.0 MB/s | 30 MB 00:07 2024-02-05T07:42:06.899 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-05T07:42:06.903 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:42:06.935 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-05T07:42:06.947 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:42:06.947 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:42:06.978 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-05T07:42:07.029 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-05T07:42:07.056 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:19.0.0-1241.g164d3be4.el9.x86_64 5/6 2024-02-05T07:42:07.397 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:42:07.397 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:42:08.722 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:42:08.776 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-02-05T07:42:08.808 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-02-05T07:42:08.837 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-02-05T07:42:08.929 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-02-05T07:42:08.969 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-02-05T07:42:09.002 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-02-05T07:42:09.093 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-02-05T07:42:09.172 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-02-05T07:42:09.213 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-02-05T07:42:09.285 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-02-05T07:42:09.338 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-02-05T07:42:09.377 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-02-05T07:42:09.427 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-02-05T07:42:09.489 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-02-05T07:42:09.674 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-02-05T07:42:09.723 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-02-05T07:42:09.763 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-02-05T07:42:09.797 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-02-05T07:42:09.827 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-02-05T07:42:09.903 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-02-05T07:42:09.945 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-05T07:42:09.979 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-02-05T07:42:10.019 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-02-05T07:42:10.054 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-02-05T07:42:10.097 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-02-05T07:42:10.186 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-02-05T07:42:10.220 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-02-05T07:42:10.293 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-02-05T07:42:10.437 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-02-05T07:42:10.564 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-05T07:42:10.721 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1241.g164d3be4.el9. 31/36 2024-02-05T07:42:10.750 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 32/36 2024-02-05T07:42:10.774 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 32/36 2024-02-05T07:42:10.774 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:42:10.774 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-05T07:42:10.774 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-05T07:42:10.774 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-05T07:42:10.774 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:11.524 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1241.g164d3be4.el9.x86_64 5/6 2024-02-05T07:42:11.655 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:19.0.0-1241.g164d3be4.el9.x86_64 6/6 2024-02-05T07:42:12.201 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-osd-2:19.0.0-1241.g164d3be4.el9.x86_64 33/36 2024-02-05T07:42:12.228 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-osd-2:19.0.0-1241.g164d3be4.el9.x86_64 33/36 2024-02-05T07:42:12.228 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:42:12.228 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-05T07:42:12.228 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-05T07:42:12.228 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-05T07:42:12.228 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:12.694 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mon-2:19.0.0-1241.g164d3be4.el9.x86_64 34/36 2024-02-05T07:42:12.719 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mon-2:19.0.0-1241.g164d3be4.el9.x86_64 34/36 2024-02-05T07:42:12.719 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:42:12.719 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-05T07:42:12.719 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-05T07:42:12.719 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-05T07:42:12.719 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:12.936 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mds-2:19.0.0-1241.g164d3be4.el9.x86_64 35/36 2024-02-05T07:42:12.960 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mds-2:19.0.0-1241.g164d3be4.el9.x86_64 35/36 2024-02-05T07:42:12.960 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:42:12.960 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-05T07:42:12.960 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-05T07:42:12.960 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-05T07:42:12.960 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:13.036 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-2:19.0.0-1241.g164d3be4.el9.x86_64 36/36 2024-02-05T07:42:13.846 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-test-2:19.0.0-1241.g164d3be4.el9.x86_64 6/6 2024-02-05T07:42:13.846 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:19.0.0-1241.g164d3be4.el9.x86_64 1/6 2024-02-05T07:42:13.846 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:19.0.0-1241.g164d3be4.el9.x86_64 2/6 2024-02-05T07:42:13.846 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-05T07:42:13.846 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-05T07:42:13.846 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-05T07:42:14.246 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-05T07:42:14.246 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:14.246 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:42:14.246 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:42:14.246 INFO:teuthology.orchestra.run.smithi157.stdout: jq-1.6-16.el9.x86_64 2024-02-05T07:42:14.247 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:42:14.247 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-05T07:42:14.247 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-05T07:42:14.247 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-05T07:42:14.247 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:14.247 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:42:14.551 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph 2024-02-05T07:42:14.595 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-2:19.0.0-1241.g164d3be4.el9.x86_64 36/36 2024-02-05T07:42:14.595 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-2:19.0.0-1241.g164d3be4.el9.x86_64 1/36 2024-02-05T07:42:14.595 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mds-2:19.0.0-1241.g164d3be4.el9.x86_64 2/36 2024-02-05T07:42:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 3/36 2024-02-05T07:42:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mon-2:19.0.0-1241.g164d3be4.el9.x86_64 4/36 2024-02-05T07:42:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-osd-2:19.0.0-1241.g164d3be4.el9.x86_64 5/36 2024-02-05T07:42:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1241.g164d3be4.el9. 6/36 2024-02-05T07:42:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-02-05T07:42:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-02-05T07:42:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-02-05T07:42:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-02-05T07:42:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-02-05T07:42:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-02-05T07:42:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-02-05T07:42:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-02-05T07:42:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-02-05T07:42:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/36 2024-02-05T07:42:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 17/36 2024-02-05T07:42:14.597 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-02-05T07:42:14.597 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-02-05T07:42:14.597 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-02-05T07:42:14.597 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-05T07:42:14.597 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-02-05T07:42:14.597 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-02-05T07:42:14.597 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-02-05T07:42:14.597 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-02-05T07:42:14.598 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-02-05T07:42:14.598 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-02-05T07:42:14.598 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-02-05T07:42:14.598 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-02-05T07:42:14.598 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-05T07:42:14.598 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-02-05T07:42:14.598 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-02-05T07:42:14.598 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-02-05T07:42:14.599 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-02-05T07:42:14.599 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-02-05T07:42:15.119 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:54 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:42:15.284 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:42:15.286 INFO:teuthology.orchestra.run.smithi157.stdout:========================================================================================= 2024-02-05T07:42:15.286 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-02-05T07:42:15.286 INFO:teuthology.orchestra.run.smithi157.stdout:========================================================================================= 2024-02-05T07:42:15.286 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:42:15.286 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 6.6 k 2024-02-05T07:42:15.287 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-02-05T07:42:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 2.1 M 2024-02-05T07:42:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 1.5 M 2024-02-05T07:42:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 247 k 2024-02-05T07:42:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 4.7 M 2024-02-05T07:42:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 17 M 2024-02-05T07:42:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-05T07:42:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-05T07:42:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-05T07:42:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-05T07:42:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-05T07:42:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-05T07:42:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-05T07:42:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-05T07:42:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-05T07:42:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-05T07:42:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-05T07:42:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-05T07:42:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-05T07:42:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-05T07:42:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-05T07:42:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-05T07:42:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-05T07:42:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-05T07:42:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-05T07:42:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-05T07:42:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-05T07:42:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-05T07:42:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-05T07:42:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-05T07:42:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-05T07:42:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-05T07:42:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-05T07:42:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-05T07:42:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-05T07:42:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-05T07:42:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:15.289 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:42:15.289 INFO:teuthology.orchestra.run.smithi157.stdout:========================================================================================= 2024-02-05T07:42:15.289 INFO:teuthology.orchestra.run.smithi157.stdout:Install 36 Packages 2024-02-05T07:42:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:15.290 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 30 M 2024-02-05T07:42:15.290 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 107 M 2024-02-05T07:42:15.290 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:42:15.613 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-02-05T07:42:15.613 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:15.613 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:42:15.613 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:42:15.613 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:42:15.614 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:42:15.614 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:42:15.614 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:42:15.614 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:42:15.614 INFO:teuthology.orchestra.run.smithi131.stdout: lua-5.4.4-4.el9.x86_64 2024-02-05T07:42:15.614 INFO:teuthology.orchestra.run.smithi131.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-05T07:42:15.614 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-05T07:42:15.614 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-05T07:42:15.614 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-05T07:42:15.614 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-05T07:42:15.614 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-05T07:42:15.614 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-05T07:42:15.614 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-05T07:42:15.614 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-05T07:42:15.614 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply-3.11-14.el9.noarch 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-05T07:42:15.615 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-05T07:42:15.616 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-05T07:42:15.616 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-05T07:42:15.616 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:15.616 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:42:15.785 INFO:teuthology.orchestra.run.smithi157.stdout:(1/36): ceph-19.0.0-1241.g164d3be4.el9.x86_64.r 43 kB/s | 6.6 kB 00:00 2024-02-05T07:42:15.914 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-base 2024-02-05T07:42:16.243 INFO:teuthology.orchestra.run.smithi157.stdout:(2/36): ceph-mon-19.0.0-1241.g164d3be4.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-02-05T07:42:16.277 INFO:teuthology.orchestra.run.smithi157.stdout:(3/36): ceph-mgr-19.0.0-1241.g164d3be4.el9.x86_ 2.3 MB/s | 1.5 MB 00:00 2024-02-05T07:42:16.319 INFO:teuthology.orchestra.run.smithi157.stdout:(4/36): ceph-mds-19.0.0-1241.g164d3be4.el9.x86_ 3.1 MB/s | 2.1 MB 00:00 2024-02-05T07:42:16.352 INFO:teuthology.orchestra.run.smithi157.stdout:(5/36): ceph-mgr-modules-core-19.0.0-1241.g164d 3.2 MB/s | 247 kB 00:00 2024-02-05T07:42:16.480 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:13 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:42:16.502 INFO:teuthology.orchestra.run.smithi157.stdout:(6/36): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-02-05T07:42:16.528 INFO:teuthology.orchestra.run.smithi157.stdout:(7/36): python3-mako-1.1.4-6.el9.noarch.rpm 952 kB/s | 144 kB 00:00 2024-02-05T07:42:16.545 INFO:teuthology.orchestra.run.smithi157.stdout:(8/36): python3-markupsafe-1.1.1-12.el9.x86_64. 756 kB/s | 32 kB 00:00 2024-02-05T07:42:16.561 INFO:teuthology.orchestra.run.smithi157.stdout:(9/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-02-05T07:42:16.578 INFO:teuthology.orchestra.run.smithi157.stdout:(10/36): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-02-05T07:42:16.585 INFO:teuthology.orchestra.run.smithi131.stdout:Package ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:42:16.612 INFO:teuthology.orchestra.run.smithi157.stdout:(11/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-02-05T07:42:16.638 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:42:16.639 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-02-05T07:42:16.639 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:42:16.671 INFO:teuthology.orchestra.run.smithi157.stdout:(12/36): python3-cryptography-36.0.1-4.el9.x86_ 12 MB/s | 1.1 MB 00:00 2024-02-05T07:42:16.696 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install cephadm 2024-02-05T07:42:16.696 INFO:teuthology.orchestra.run.smithi157.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-02-05T07:42:16.721 INFO:teuthology.orchestra.run.smithi157.stdout:(14/36): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-02-05T07:42:16.747 INFO:teuthology.orchestra.run.smithi157.stdout:(15/36): python3-requests-2.25.1-8.el9.noarch.r 2.2 MB/s | 113 kB 00:00 2024-02-05T07:42:16.772 INFO:teuthology.orchestra.run.smithi157.stdout:(16/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-02-05T07:42:16.906 INFO:teuthology.orchestra.run.smithi157.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2024-02-05T07:42:17.123 INFO:teuthology.orchestra.run.smithi157.stdout:(18/36): ceph-osd-19.0.0-1241.g164d3be4.el9.x86 19 MB/s | 17 MB 00:00 2024-02-05T07:42:17.147 INFO:teuthology.orchestra.run.smithi157.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 402 kB/s | 151 kB 00:00 2024-02-05T07:42:17.173 INFO:teuthology.orchestra.run.smithi157.stdout:(20/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 163 kB/s | 43 kB 00:00 2024-02-05T07:42:17.260 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:14 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:42:17.418 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:42:17.419 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:42:17.419 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-02-05T07:42:17.419 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:42:17.419 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:42:17.419 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 748 k 2024-02-05T07:42:17.419 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:17.419 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:42:17.419 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:42:17.420 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-02-05T07:42:17.420 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:17.420 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 748 k 2024-02-05T07:42:17.420 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 754 k 2024-02-05T07:42:17.420 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:42:17.711 INFO:teuthology.orchestra.run.smithi131.stdout:cephadm-19.0.0-1241.g164d3be4.el9.noarch.rpm 2.5 MB/s | 748 kB 00:00 2024-02-05T07:42:17.711 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:42:17.711 INFO:teuthology.orchestra.run.smithi131.stdout:Total 2.5 MB/s | 748 kB 00:00 2024-02-05T07:42:17.712 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:42:17.716 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:42:17.716 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:42:17.722 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:42:17.722 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:42:17.798 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:42:17.973 INFO:teuthology.orchestra.run.smithi157.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 202 kB/s | 172 kB 00:00 2024-02-05T07:42:18.252 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/1 2024-02-05T07:42:18.323 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/1 2024-02-05T07:42:18.523 INFO:teuthology.orchestra.run.smithi157.stdout:(22/36): python3-cherrypy-18.6.1-2.el9.noarch.r 265 kB/s | 358 kB 00:01 2024-02-05T07:42:18.549 INFO:teuthology.orchestra.run.smithi157.stdout:(23/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 7.8 kB/s | 11 kB 00:01 2024-02-05T07:42:18.574 INFO:teuthology.orchestra.run.smithi157.stdout:(24/36): python3-jaraco-classes-3.2.1-5.el9.noa 29 kB/s | 18 kB 00:00 2024-02-05T07:42:18.616 INFO:teuthology.orchestra.run.smithi157.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 251 kB/s | 23 kB 00:00 2024-02-05T07:42:18.708 INFO:teuthology.orchestra.run.smithi157.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 146 kB/s | 19 kB 00:00 2024-02-05T07:42:18.752 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/1 2024-02-05T07:42:18.808 INFO:teuthology.orchestra.run.smithi157.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 84 kB/s | 20 kB 00:00 2024-02-05T07:42:18.958 INFO:teuthology.orchestra.run.smithi157.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 135 kB/s | 46 kB 00:00 2024-02-05T07:42:19.082 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/1 2024-02-05T07:42:19.082 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:19.082 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:42:19.082 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:42:19.082 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:19.082 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:42:19.291 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-immutable-object-cache 2024-02-05T07:42:19.292 INFO:teuthology.orchestra.run.smithi157.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 135 kB/s | 79 kB 00:00 2024-02-05T07:42:19.867 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:16 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:42:20.025 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:42:20.026 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:42:20.026 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repo Size 2024-02-05T07:42:20.026 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:42:20.026 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:42:20.027 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 145 k 2024-02-05T07:42:20.027 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:20.027 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:42:20.027 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:42:20.027 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-02-05T07:42:20.027 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:20.027 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 145 k 2024-02-05T07:42:20.027 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 436 k 2024-02-05T07:42:20.027 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:42:20.242 INFO:teuthology.orchestra.run.smithi157.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 190 kB/s | 272 kB 00:01 2024-02-05T07:42:20.351 INFO:teuthology.orchestra.run.smithi157.stdout:(31/36): python3-portend-3.1.0-2.el9.noarch.rpm 12 kB/s | 16 kB 00:01 2024-02-05T07:42:20.433 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-immutable-object-cache-19.0.0-1241.g164d3b 357 kB/s | 145 kB 00:00 2024-02-05T07:42:20.434 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:42:20.434 INFO:teuthology.orchestra.run.smithi131.stdout:Total 356 kB/s | 145 kB 00:00 2024-02-05T07:42:20.435 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:42:20.442 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:42:20.442 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:42:20.495 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:42:20.495 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:42:20.593 INFO:teuthology.orchestra.run.smithi157.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 69 kB/s | 90 kB 00:01 2024-02-05T07:42:20.643 INFO:teuthology.orchestra.run.smithi157.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 89 kB/s | 36 kB 00:00 2024-02-05T07:42:20.655 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:42:20.709 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1241.g164d3be4. 1/1 2024-02-05T07:42:20.734 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1241.g164d3be4. 1/1 2024-02-05T07:42:20.734 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:42:20.734 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-05T07:42:20.734 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:20.993 INFO:teuthology.orchestra.run.smithi157.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 358 kB/s | 230 kB 00:00 2024-02-05T07:42:21.593 INFO:teuthology.orchestra.run.smithi157.stdout:(35/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 427 kB/s | 427 kB 00:01 2024-02-05T07:42:21.635 INFO:teuthology.orchestra.run.smithi157.stdout:(36/36): python3-zc-lockfile-2.0-10.el9.noarch. 20 kB/s | 20 kB 00:00 2024-02-05T07:42:21.641 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:42:21.641 INFO:teuthology.orchestra.run.smithi157.stdout:Total 4.7 MB/s | 30 MB 00:06 2024-02-05T07:42:21.662 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:42:21.706 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:42:21.706 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:42:21.709 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1241.g164d3be4. 1/1 2024-02-05T07:42:21.709 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:21.709 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:42:21.709 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:42:21.709 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:21.709 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:42:21.920 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr 2024-02-05T07:42:22.129 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:42:22.130 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:42:22.499 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:19 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:42:22.605 INFO:teuthology.orchestra.run.smithi131.stdout:Package ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:42:22.655 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:42:22.656 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-02-05T07:42:22.656 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:42:22.712 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr-dashboard 2024-02-05T07:42:23.279 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:20 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:42:23.437 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:42:23.439 INFO:teuthology.orchestra.run.smithi131.stdout:====================================================================================== 2024-02-05T07:42:23.439 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-02-05T07:42:23.439 INFO:teuthology.orchestra.run.smithi131.stdout:====================================================================================== 2024-02-05T07:42:23.439 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:42:23.439 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 1.8 M 2024-02-05T07:42:23.439 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-02-05T07:42:23.439 INFO:teuthology.orchestra.run.smithi131.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-05T07:42:23.440 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 28 k 2024-02-05T07:42:23.440 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 15 k 2024-02-05T07:42:23.440 INFO:teuthology.orchestra.run.smithi131.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-05T07:42:23.440 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-05T07:42:23.440 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-05T07:42:23.440 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-05T07:42:23.440 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-05T07:42:23.440 INFO:teuthology.orchestra.run.smithi131.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-05T07:42:23.440 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-05T07:42:23.440 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-05T07:42:23.440 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:23.440 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:42:23.441 INFO:teuthology.orchestra.run.smithi131.stdout:====================================================================================== 2024-02-05T07:42:23.441 INFO:teuthology.orchestra.run.smithi131.stdout:Install 12 Packages 2024-02-05T07:42:23.441 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:23.441 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:42:23.442 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 6.8 M 2024-02-05T07:42:23.442 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 26 M 2024-02-05T07:42:23.442 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:42:23.488 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-02-05T07:42:23.528 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-02-05T07:42:23.565 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-02-05T07:42:23.635 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-02-05T07:42:23.672 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-02-05T07:42:23.705 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-02-05T07:42:23.784 INFO:teuthology.orchestra.run.smithi131.stdout:(1/12): ceph-prometheus-alerts-19.0.0-1241.g164 81 kB/s | 15 kB 00:00 2024-02-05T07:42:23.797 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-02-05T07:42:23.851 INFO:teuthology.orchestra.run.smithi131.stdout:(2/12): ceph-grafana-dashboards-19.0.0-1241.g16 111 kB/s | 28 kB 00:00 2024-02-05T07:42:23.867 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-02-05T07:42:23.907 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-02-05T07:42:23.976 INFO:teuthology.orchestra.run.smithi131.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.9 MB/s | 238 kB 00:00 2024-02-05T07:42:23.980 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-02-05T07:42:24.009 INFO:teuthology.orchestra.run.smithi131.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-02-05T07:42:24.033 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-02-05T07:42:24.072 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-02-05T07:42:24.121 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-02-05T07:42:24.182 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-02-05T07:42:24.235 INFO:teuthology.orchestra.run.smithi131.stdout:(5/12): ceph-mgr-dashboard-19.0.0-1241.g164d3be 2.7 MB/s | 1.8 MB 00:00 2024-02-05T07:42:24.319 INFO:teuthology.orchestra.run.smithi131.stdout:(6/12): protobuf-compiler-3.14.0-13.el9.x86_64. 2.5 MB/s | 863 kB 00:00 2024-02-05T07:42:24.370 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-02-05T07:42:24.426 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-02-05T07:42:24.474 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-02-05T07:42:24.508 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-02-05T07:42:24.538 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-02-05T07:42:24.614 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-02-05T07:42:24.648 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-05T07:42:24.681 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-02-05T07:42:24.714 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-02-05T07:42:24.749 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-02-05T07:42:24.791 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-02-05T07:42:24.879 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-02-05T07:42:24.922 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-02-05T07:42:24.993 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-02-05T07:42:25.133 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-02-05T07:42:25.256 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-05T07:42:25.422 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1241.g164d3be4.el9. 31/36 2024-02-05T07:42:25.452 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 32/36 2024-02-05T07:42:25.478 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 32/36 2024-02-05T07:42:25.478 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:42:25.478 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-05T07:42:25.478 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-05T07:42:25.478 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-05T07:42:25.478 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:26.627 INFO:teuthology.orchestra.run.smithi131.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 8.1 kB/s | 19 kB 00:02 2024-02-05T07:42:26.836 INFO:teuthology.orchestra.run.smithi131.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 194 kB/s | 548 kB 00:02 2024-02-05T07:42:26.911 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:19.0.0-1241.g164d3be4.el9.x86_64 33/36 2024-02-05T07:42:26.937 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:19.0.0-1241.g164d3be4.el9.x86_64 33/36 2024-02-05T07:42:26.937 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:42:26.937 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-05T07:42:26.938 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-05T07:42:26.938 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-05T07:42:26.938 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:27.378 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:19.0.0-1241.g164d3be4.el9.x86_64 34/36 2024-02-05T07:42:27.402 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:19.0.0-1241.g164d3be4.el9.x86_64 34/36 2024-02-05T07:42:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:42:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-05T07:42:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-05T07:42:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-05T07:42:27.473 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:27.621 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:19.0.0-1241.g164d3be4.el9.x86_64 35/36 2024-02-05T07:42:27.644 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:19.0.0-1241.g164d3be4.el9.x86_64 35/36 2024-02-05T07:42:27.644 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:42:27.644 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-05T07:42:27.644 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-05T07:42:27.644 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-05T07:42:27.644 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:27.710 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:19.0.0-1241.g164d3be4.el9.x86_64 36/36 2024-02-05T07:42:29.111 INFO:teuthology.orchestra.run.smithi131.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 14 kB/s | 31 kB 00:02 2024-02-05T07:42:29.565 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-2:19.0.0-1241.g164d3be4.el9.x86_64 36/36 2024-02-05T07:42:29.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:19.0.0-1241.g164d3be4.el9.x86_64 1/36 2024-02-05T07:42:29.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:19.0.0-1241.g164d3be4.el9.x86_64 2/36 2024-02-05T07:42:29.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 3/36 2024-02-05T07:42:29.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:19.0.0-1241.g164d3be4.el9.x86_64 4/36 2024-02-05T07:42:29.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:19.0.0-1241.g164d3be4.el9.x86_64 5/36 2024-02-05T07:42:29.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1241.g164d3be4.el9. 6/36 2024-02-05T07:42:29.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-02-05T07:42:29.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-02-05T07:42:29.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-02-05T07:42:29.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-02-05T07:42:29.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-02-05T07:42:29.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-02-05T07:42:29.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-02-05T07:42:29.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-02-05T07:42:29.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-02-05T07:42:29.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/36 2024-02-05T07:42:29.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 17/36 2024-02-05T07:42:29.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-02-05T07:42:29.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-02-05T07:42:29.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-02-05T07:42:29.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-05T07:42:29.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-02-05T07:42:29.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-02-05T07:42:29.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-02-05T07:42:29.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-02-05T07:42:29.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-02-05T07:42:29.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-02-05T07:42:29.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-02-05T07:42:29.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-02-05T07:42:29.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-05T07:42:29.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-02-05T07:42:29.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-02-05T07:42:29.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-02-05T07:42:29.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-02-05T07:42:29.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-02-05T07:42:30.244 INFO:teuthology.orchestra.run.smithi131.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 40 kB/s | 144 kB 00:03 2024-02-05T07:42:30.403 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-02-05T07:42:30.403 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:30.403 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:42:30.403 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:42:30.403 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:42:30.404 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:42:30.404 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:42:30.404 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:42:30.404 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:42:30.404 INFO:teuthology.orchestra.run.smithi157.stdout: lua-5.4.4-4.el9.x86_64 2024-02-05T07:42:30.404 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-05T07:42:30.404 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-05T07:42:30.404 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-05T07:42:30.404 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-05T07:42:30.404 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-05T07:42:30.404 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-05T07:42:30.404 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-05T07:42:30.404 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-05T07:42:30.404 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-05T07:42:30.405 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-05T07:42:30.405 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-05T07:42:30.405 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-05T07:42:30.405 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-05T07:42:30.405 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-05T07:42:30.405 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-05T07:42:30.405 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-05T07:42:30.405 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-05T07:42:30.405 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply-3.11-14.el9.noarch 2024-02-05T07:42:30.405 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-05T07:42:30.405 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-05T07:42:30.405 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-05T07:42:30.405 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-05T07:42:30.405 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-05T07:42:30.405 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-05T07:42:30.406 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-05T07:42:30.406 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-05T07:42:30.406 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-05T07:42:30.406 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-05T07:42:30.406 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-05T07:42:30.406 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:30.406 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:42:30.744 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-base 2024-02-05T07:42:31.317 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:10 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:42:31.424 INFO:teuthology.orchestra.run.smithi157.stdout:Package ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:42:31.475 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:42:31.476 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-02-05T07:42:31.476 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:42:31.539 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install cephadm 2024-02-05T07:42:32.103 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:11 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:42:32.259 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:42:32.260 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:42:32.260 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-02-05T07:42:32.260 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:42:32.260 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:42:32.260 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 748 k 2024-02-05T07:42:32.260 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:32.260 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:42:32.261 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:42:32.261 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-02-05T07:42:32.261 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:32.261 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 748 k 2024-02-05T07:42:32.261 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 754 k 2024-02-05T07:42:32.261 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:42:32.594 INFO:teuthology.orchestra.run.smithi131.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 54 kB/s | 188 kB 00:03 2024-02-05T07:42:32.719 INFO:teuthology.orchestra.run.smithi157.stdout:cephadm-19.0.0-1241.g164d3be4.el9.noarch.rpm 1.6 MB/s | 748 kB 00:00 2024-02-05T07:42:32.719 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:42:32.720 INFO:teuthology.orchestra.run.smithi157.stdout:Total 1.6 MB/s | 748 kB 00:00 2024-02-05T07:42:32.720 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:42:32.725 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:42:32.725 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:42:32.731 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:42:32.731 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:42:32.805 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:42:33.235 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/1 2024-02-05T07:42:33.324 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/1 2024-02-05T07:42:33.729 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/1 2024-02-05T07:42:34.027 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/1 2024-02-05T07:42:34.027 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:34.027 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:42:34.027 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:42:34.027 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:34.027 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:42:34.227 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-immutable-object-cache 2024-02-05T07:42:34.352 INFO:teuthology.orchestra.run.smithi131.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 208 kB/s | 2.0 MB 00:10 2024-02-05T07:42:34.356 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:42:34.356 INFO:teuthology.orchestra.run.smithi131.stdout:Total 641 kB/s | 6.8 MB 00:10 2024-02-05T07:42:34.376 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:42:34.395 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:42:34.395 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:42:34.598 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:42:34.599 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:42:34.806 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:13 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:42:34.962 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:42:34.962 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:42:34.962 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repo Size 2024-02-05T07:42:34.962 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:42:34.962 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:42:34.962 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 145 k 2024-02-05T07:42:34.962 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:34.962 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:42:34.962 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:42:34.962 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-02-05T07:42:34.963 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:34.963 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 145 k 2024-02-05T07:42:34.963 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 436 k 2024-02-05T07:42:34.963 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:42:35.115 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:42:35.194 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-02-05T07:42:35.238 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-02-05T07:42:35.284 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-02-05T07:42:35.315 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-02-05T07:42:35.324 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-immutable-object-cache-19.0.0-1241.g164d3b 403 kB/s | 145 kB 00:00 2024-02-05T07:42:35.325 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:42:35.325 INFO:teuthology.orchestra.run.smithi157.stdout:Total 401 kB/s | 145 kB 00:00 2024-02-05T07:42:35.325 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:42:35.333 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:42:35.333 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:42:35.386 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:42:35.387 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:42:35.414 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-02-05T07:42:35.551 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-02-05T07:42:35.557 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:42:35.617 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1241.g164d3be4. 1/1 2024-02-05T07:42:35.634 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-02-05T07:42:35.642 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1241.g164d3be4. 1/1 2024-02-05T07:42:35.642 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:42:35.642 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-05T07:42:35.642 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:35.686 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-02-05T07:42:35.715 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-02-05T07:42:35.745 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1241.g164d3be4.el9 10/12 2024-02-05T07:42:35.991 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1241.g164d3be4.el 11/12 2024-02-05T07:42:36.030 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1241.g164d3be4.el9.noa 12/12 2024-02-05T07:42:36.635 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1241.g164d3be4. 1/1 2024-02-05T07:42:36.636 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:36.636 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:42:36.636 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:42:36.636 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:36.636 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:42:36.869 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1241.g164d3be4.el9.noa 12/12 2024-02-05T07:42:36.870 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1241.g164d3be4.el 1/12 2024-02-05T07:42:36.870 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1241.g164d3be4.el9.noa 2/12 2024-02-05T07:42:36.870 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1241.g164d3be4.el9 3/12 2024-02-05T07:42:36.870 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-02-05T07:42:36.870 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-02-05T07:42:36.870 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-02-05T07:42:36.870 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-02-05T07:42:36.870 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-02-05T07:42:36.870 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-02-05T07:42:36.870 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-02-05T07:42:36.870 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-02-05T07:42:36.878 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr 2024-02-05T07:42:37.219 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-02-05T07:42:37.219 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:37.219 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:42:37.220 INFO:teuthology.orchestra.run.smithi131.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-05T07:42:37.220 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:42:37.220 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:42:37.220 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:42:37.220 INFO:teuthology.orchestra.run.smithi131.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-05T07:42:37.220 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-05T07:42:37.220 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-05T07:42:37.220 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-05T07:42:37.220 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-05T07:42:37.220 INFO:teuthology.orchestra.run.smithi131.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-05T07:42:37.220 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-05T07:42:37.220 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-05T07:42:37.220 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:37.220 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:42:37.456 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:16 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:42:37.464 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-05T07:42:37.560 INFO:teuthology.orchestra.run.smithi157.stdout:Package ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:42:37.612 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:42:37.613 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-02-05T07:42:37.613 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:42:37.688 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr-dashboard 2024-02-05T07:42:38.028 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:35 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:42:38.184 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:42:38.186 INFO:teuthology.orchestra.run.smithi131.stdout:============================================================================================ 2024-02-05T07:42:38.186 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-02-05T07:42:38.186 INFO:teuthology.orchestra.run.smithi131.stdout:============================================================================================ 2024-02-05T07:42:38.186 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:42:38.186 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 7.4 M 2024-02-05T07:42:38.186 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-02-05T07:42:38.186 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-05T07:42:38.186 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-05T07:42:38.186 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-05T07:42:38.186 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-05T07:42:38.186 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-05T07:42:38.187 INFO:teuthology.orchestra.run.smithi131.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-05T07:42:38.187 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-05T07:42:38.187 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-05T07:42:38.187 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-05T07:42:38.187 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-05T07:42:38.187 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-05T07:42:38.187 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:38.187 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:42:38.187 INFO:teuthology.orchestra.run.smithi131.stdout:============================================================================================ 2024-02-05T07:42:38.187 INFO:teuthology.orchestra.run.smithi131.stdout:Install 12 Packages 2024-02-05T07:42:38.187 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:38.188 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 37 M 2024-02-05T07:42:38.188 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 209 M 2024-02-05T07:42:38.188 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:42:38.268 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:17 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:42:38.438 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:42:38.439 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================== 2024-02-05T07:42:38.439 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-02-05T07:42:38.439 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================== 2024-02-05T07:42:38.439 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:42:38.439 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 1.8 M 2024-02-05T07:42:38.439 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-02-05T07:42:38.439 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-05T07:42:38.439 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 28 k 2024-02-05T07:42:38.439 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 15 k 2024-02-05T07:42:38.440 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-05T07:42:38.440 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-05T07:42:38.440 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-05T07:42:38.440 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-05T07:42:38.440 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-05T07:42:38.440 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-05T07:42:38.440 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-05T07:42:38.440 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-05T07:42:38.440 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:38.440 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:42:38.440 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================== 2024-02-05T07:42:38.440 INFO:teuthology.orchestra.run.smithi157.stdout:Install 12 Packages 2024-02-05T07:42:38.440 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:38.441 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 6.8 M 2024-02-05T07:42:38.441 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 26 M 2024-02-05T07:42:38.441 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:42:38.609 INFO:teuthology.orchestra.run.smithi131.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 209 kB/s | 32 kB 00:00 2024-02-05T07:42:38.651 INFO:teuthology.orchestra.run.smithi131.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 415 kB/s | 17 kB 00:00 2024-02-05T07:42:38.701 INFO:teuthology.orchestra.run.smithi131.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 695 kB/s | 34 kB 00:00 2024-02-05T07:42:38.751 INFO:teuthology.orchestra.run.smithi131.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 10 MB/s | 3.0 MB 00:00 2024-02-05T07:42:38.944 INFO:teuthology.orchestra.run.smithi131.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2024-02-05T07:42:39.027 INFO:teuthology.orchestra.run.smithi131.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2024-02-05T07:42:39.053 INFO:teuthology.orchestra.run.smithi131.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 217 kB 00:00 2024-02-05T07:42:39.110 INFO:teuthology.orchestra.run.smithi157.stdout:(1/12): ceph-prometheus-alerts-19.0.0-1241.g164 81 kB/s | 15 kB 00:00 2024-02-05T07:42:39.186 INFO:teuthology.orchestra.run.smithi131.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 5.9 MB/s | 802 kB 00:00 2024-02-05T07:42:39.219 INFO:teuthology.orchestra.run.smithi157.stdout:(2/12): ceph-grafana-dashboards-19.0.0-1241.g16 95 kB/s | 28 kB 00:00 2024-02-05T07:42:39.344 INFO:teuthology.orchestra.run.smithi157.stdout:(3/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-02-05T07:42:39.361 INFO:teuthology.orchestra.run.smithi131.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 47 MB/s | 16 MB 00:00 2024-02-05T07:42:39.370 INFO:teuthology.orchestra.run.smithi157.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.5 MB/s | 238 kB 00:00 2024-02-05T07:42:39.387 INFO:teuthology.orchestra.run.smithi131.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 981 kB/s | 194 kB 00:00 2024-02-05T07:42:39.470 INFO:teuthology.orchestra.run.smithi131.stdout:(11/12): ceph-mgr-diskprediction-local-19.0.0-1 7.3 MB/s | 7.4 MB 00:01 2024-02-05T07:42:39.520 INFO:teuthology.orchestra.run.smithi157.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 3.6 MB/s | 548 kB 00:00 2024-02-05T07:42:39.562 INFO:teuthology.orchestra.run.smithi157.stdout:(6/12): ceph-mgr-dashboard-19.0.0-1241.g164d3be 2.7 MB/s | 1.8 MB 00:00 2024-02-05T07:42:39.620 INFO:teuthology.orchestra.run.smithi157.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-02-05T07:42:39.662 INFO:teuthology.orchestra.run.smithi157.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 3.4 MB/s | 144 kB 00:00 2024-02-05T07:42:39.670 INFO:teuthology.orchestra.run.smithi131.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 796 kB/s | 245 kB 00:00 2024-02-05T07:42:39.673 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:42:39.674 INFO:teuthology.orchestra.run.smithi131.stdout:Total 25 MB/s | 37 MB 00:01 2024-02-05T07:42:39.686 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:42:39.704 INFO:teuthology.orchestra.run.smithi157.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 739 kB/s | 31 kB 00:00 2024-02-05T07:42:39.711 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:42:39.711 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:42:39.729 INFO:teuthology.orchestra.run.smithi157.stdout:(10/12): protobuf-compiler-3.14.0-13.el9.x86_64 2.2 MB/s | 863 kB 00:00 2024-02-05T07:42:39.746 INFO:teuthology.orchestra.run.smithi157.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 4.4 MB/s | 188 kB 00:00 2024-02-05T07:42:39.805 INFO:teuthology.orchestra.run.smithi157.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 8.4 MB/s | 2.0 MB 00:00 2024-02-05T07:42:39.808 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:42:39.808 INFO:teuthology.orchestra.run.smithi157.stdout:Total 5.0 MB/s | 6.8 MB 00:01 2024-02-05T07:42:39.825 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:42:39.843 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:42:39.843 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:42:40.032 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:42:40.032 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:42:40.069 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:42:40.069 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:42:40.551 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:42:40.639 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-02-05T07:42:40.674 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-02-05T07:42:40.721 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-02-05T07:42:40.751 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-02-05T07:42:40.784 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:42:40.849 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-02-05T07:42:40.861 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-05T07:42:40.891 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-05T07:42:40.978 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-02-05T07:42:40.985 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-05T07:42:41.028 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-05T07:42:41.071 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-02-05T07:42:41.114 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-02-05T07:42:41.142 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-02-05T07:42:41.172 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1241.g164d3be4.el9 10/12 2024-02-05T07:42:41.205 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-05T07:42:41.244 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-05T07:42:41.320 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-05T07:42:41.407 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1241.g164d3be4.el 11/12 2024-02-05T07:42:41.440 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1241.g164d3be4.el9.noa 12/12 2024-02-05T07:42:41.979 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-05T07:42:42.091 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-05T07:42:42.226 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1241.g164d3be4.el9.noa 12/12 2024-02-05T07:42:42.226 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1241.g164d3be4.el 1/12 2024-02-05T07:42:42.226 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1241.g164d3be4.el9.noa 2/12 2024-02-05T07:42:42.226 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1241.g164d3be4.el9 3/12 2024-02-05T07:42:42.226 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-02-05T07:42:42.226 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-02-05T07:42:42.226 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-02-05T07:42:42.226 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-02-05T07:42:42.227 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-02-05T07:42:42.227 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-02-05T07:42:42.227 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-02-05T07:42:42.227 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-02-05T07:42:42.574 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-02-05T07:42:42.574 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:42.574 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:42:42.574 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-05T07:42:42.575 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:42:42.575 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:42:42.575 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:42:42.575 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-05T07:42:42.575 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-05T07:42:42.575 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-05T07:42:42.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-05T07:42:42.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-05T07:42:42.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-05T07:42:42.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-05T07:42:42.575 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-05T07:42:42.575 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:42.576 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:42:42.815 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-05T07:42:43.336 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-05T07:42:43.386 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:22 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:42:43.553 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:42:43.554 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================================ 2024-02-05T07:42:43.555 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-02-05T07:42:43.555 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================================ 2024-02-05T07:42:43.555 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:42:43.555 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 7.4 M 2024-02-05T07:42:43.555 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-02-05T07:42:43.555 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-05T07:42:43.555 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-05T07:42:43.555 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-05T07:42:43.555 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-05T07:42:43.555 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-05T07:42:43.555 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-05T07:42:43.555 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-05T07:42:43.555 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-05T07:42:43.556 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-05T07:42:43.556 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-05T07:42:43.556 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-05T07:42:43.556 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:43.556 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:42:43.556 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================================ 2024-02-05T07:42:43.556 INFO:teuthology.orchestra.run.smithi157.stdout:Install 12 Packages 2024-02-05T07:42:43.556 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:43.557 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 37 M 2024-02-05T07:42:43.557 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 209 M 2024-02-05T07:42:43.557 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:42:43.862 INFO:teuthology.orchestra.run.smithi157.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 215 kB/s | 32 kB 00:00 2024-02-05T07:42:43.895 INFO:teuthology.orchestra.run.smithi157.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 522 kB/s | 17 kB 00:00 2024-02-05T07:42:43.962 INFO:teuthology.orchestra.run.smithi157.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 515 kB/s | 34 kB 00:00 2024-02-05T07:42:44.162 INFO:teuthology.orchestra.run.smithi157.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 6.7 MB/s | 3.0 MB 00:00 2024-02-05T07:42:44.370 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-05T07:42:44.379 INFO:teuthology.orchestra.run.smithi157.stdout:(5/12): ceph-mgr-diskprediction-local-19.0.0-12 11 MB/s | 7.4 MB 00:00 2024-02-05T07:42:44.411 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1241.g164d3 12/12 2024-02-05T07:42:44.446 INFO:teuthology.orchestra.run.smithi157.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 9.8 MB/s | 4.7 MB 00:00 2024-02-05T07:42:44.522 INFO:teuthology.orchestra.run.smithi157.stdout:(7/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 4.8 MB 00:00 2024-02-05T07:42:44.547 INFO:teuthology.orchestra.run.smithi157.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.3 MB/s | 217 kB 00:00 2024-02-05T07:42:44.864 INFO:teuthology.orchestra.run.smithi157.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-02-05T07:42:44.889 INFO:teuthology.orchestra.run.smithi157.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 572 kB/s | 194 kB 00:00 2024-02-05T07:42:44.922 INFO:teuthology.orchestra.run.smithi157.stdout:(11/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 802 kB 00:00 2024-02-05T07:42:45.681 INFO:teuthology.orchestra.run.smithi157.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 300 kB/s | 245 kB 00:00 2024-02-05T07:42:45.684 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:42:45.685 INFO:teuthology.orchestra.run.smithi157.stdout:Total 17 MB/s | 37 MB 00:02 2024-02-05T07:42:45.699 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:42:45.727 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:42:45.727 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:42:46.087 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:42:46.088 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:42:46.798 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:42:46.871 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-05T07:42:46.910 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-05T07:42:47.001 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1241.g164d3 12/12 2024-02-05T07:42:47.001 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1241.g164d3 1/12 2024-02-05T07:42:47.001 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-05T07:42:47.001 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-05T07:42:47.001 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-05T07:42:47.001 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-05T07:42:47.001 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-05T07:42:47.001 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-05T07:42:47.001 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-05T07:42:47.001 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-05T07:42:47.001 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-05T07:42:47.002 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-05T07:42:47.007 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-05T07:42:47.047 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-05T07:42:47.227 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-05T07:42:47.254 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-05T07:42:47.396 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-05T07:42:47.480 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-05T07:42:47.480 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:47.481 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:42:47.481 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:42:47.481 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-05T07:42:47.481 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-05T07:42:47.481 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-05T07:42:47.481 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-05T07:42:47.481 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-05T07:42:47.481 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-05T07:42:47.481 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-05T07:42:47.481 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-05T07:42:47.481 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-05T07:42:47.481 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-05T07:42:47.481 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-05T07:42:47.482 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:47.482 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:42:47.822 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr-rook 2024-02-05T07:42:48.125 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-05T07:42:48.233 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-05T07:42:48.400 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:45 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:42:48.557 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:42:48.558 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================== 2024-02-05T07:42:48.558 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-02-05T07:42:48.558 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================== 2024-02-05T07:42:48.558 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:42:48.559 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 52 k 2024-02-05T07:42:48.559 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-02-05T07:42:48.559 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-05T07:42:48.559 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-05T07:42:48.559 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-05T07:42:48.559 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-05T07:42:48.559 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-05T07:42:48.559 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-05T07:42:48.559 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-05T07:42:48.559 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-05T07:42:48.559 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-05T07:42:48.559 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-05T07:42:48.559 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-05T07:42:48.560 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-05T07:42:48.560 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:48.560 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:42:48.560 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================== 2024-02-05T07:42:48.560 INFO:teuthology.orchestra.run.smithi131.stdout:Install 13 Packages 2024-02-05T07:42:48.560 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:48.561 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 2.0 M 2024-02-05T07:42:48.561 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 26 M 2024-02-05T07:42:48.561 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:42:48.738 INFO:teuthology.orchestra.run.smithi131.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 181 kB/s | 24 kB 00:00 2024-02-05T07:42:48.763 INFO:teuthology.orchestra.run.smithi131.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 113 kB/s | 18 kB 00:00 2024-02-05T07:42:48.846 INFO:teuthology.orchestra.run.smithi131.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.1 MB/s | 174 kB 00:00 2024-02-05T07:42:48.872 INFO:teuthology.orchestra.run.smithi131.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 132 kB 00:00 2024-02-05T07:42:48.897 INFO:teuthology.orchestra.run.smithi131.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-02-05T07:42:48.922 INFO:teuthology.orchestra.run.smithi131.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 857 kB/s | 43 kB 00:00 2024-02-05T07:42:48.947 INFO:teuthology.orchestra.run.smithi131.stdout:(7/13): ceph-mgr-rook-19.0.0-1241.g164d3be4.el9 152 kB/s | 52 kB 00:00 2024-02-05T07:42:49.106 INFO:teuthology.orchestra.run.smithi131.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 82 kB/s | 15 kB 00:00 2024-02-05T07:42:49.131 INFO:teuthology.orchestra.run.smithi131.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 137 kB/s | 32 kB 00:00 2024-02-05T07:42:49.431 INFO:teuthology.orchestra.run.smithi131.stdout:(10/13): python3-google-auth-2.27.0-1.el9.noarc 419 kB/s | 203 kB 00:00 2024-02-05T07:42:49.468 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-05T07:42:49.840 INFO:teuthology.orchestra.run.smithi131.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 83 kB/s | 59 kB 00:00 2024-02-05T07:42:50.041 INFO:teuthology.orchestra.run.smithi131.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 147 kB/s | 90 kB 00:00 2024-02-05T07:42:50.116 INFO:teuthology.orchestra.run.smithi131.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.0 MB/s | 1.0 MB 00:01 2024-02-05T07:42:50.121 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:42:50.121 INFO:teuthology.orchestra.run.smithi131.stdout:Total 1.3 MB/s | 2.0 MB 00:01 2024-02-05T07:42:50.121 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:42:50.136 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:42:50.136 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:42:50.241 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:42:50.241 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:42:50.494 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-05T07:42:50.528 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1241.g164d3 12/12 2024-02-05T07:42:50.775 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:42:50.955 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-05T07:42:51.008 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-05T07:42:51.072 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-05T07:42:51.101 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-05T07:42:51.142 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-05T07:42:51.236 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-05T07:42:51.345 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-05T07:42:51.402 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-05T07:42:52.125 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-05T07:42:52.183 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-05T07:42:52.217 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-05T07:42:52.265 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-05T07:42:52.287 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-rook-2:19.0.0-1241.g164d3be4.el9.noarch 13/13 2024-02-05T07:42:53.058 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1241.g164d3 12/12 2024-02-05T07:42:53.058 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1241.g164d3 1/12 2024-02-05T07:42:53.059 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-05T07:42:53.059 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-05T07:42:53.059 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-05T07:42:53.059 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-05T07:42:53.059 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-05T07:42:53.059 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-05T07:42:53.059 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-05T07:42:53.059 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-05T07:42:53.059 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-05T07:42:53.059 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-05T07:42:53.112 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1241.g164d3be4.el9.noarch 13/13 2024-02-05T07:42:53.112 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1241.g164d3be4.el9.noarch 1/13 2024-02-05T07:42:53.112 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-05T07:42:53.112 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-05T07:42:53.112 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-05T07:42:53.112 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-05T07:42:53.113 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-05T07:42:53.113 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-05T07:42:53.113 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-05T07:42:53.113 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-05T07:42:53.113 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-05T07:42:53.113 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-05T07:42:53.113 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-05T07:42:53.452 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-05T07:42:53.453 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:53.453 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:42:53.453 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:42:53.453 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-05T07:42:53.453 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-05T07:42:53.453 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-05T07:42:53.453 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-05T07:42:53.453 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-05T07:42:53.453 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-05T07:42:53.453 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-05T07:42:53.453 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-05T07:42:53.453 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-05T07:42:53.453 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-05T07:42:53.453 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-05T07:42:53.454 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:53.454 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:42:53.494 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-05T07:42:53.494 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:53.494 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:42:53.495 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:42:53.495 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-05T07:42:53.495 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-05T07:42:53.495 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-05T07:42:53.495 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-05T07:42:53.495 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-05T07:42:53.495 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-05T07:42:53.495 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-05T07:42:53.495 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-05T07:42:53.495 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-05T07:42:53.495 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-05T07:42:53.495 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-05T07:42:53.495 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-05T07:42:53.496 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:53.496 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:42:53.731 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr-cephadm 2024-02-05T07:42:53.856 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr-rook 2024-02-05T07:42:54.297 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:51 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:42:54.434 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:33 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:42:54.451 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:42:54.452 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================== 2024-02-05T07:42:54.452 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-02-05T07:42:54.452 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================== 2024-02-05T07:42:54.453 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:42:54.453 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 138 k 2024-02-05T07:42:54.453 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-02-05T07:42:54.453 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-05T07:42:54.453 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-05T07:42:54.453 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-02-05T07:42:54.453 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-05T07:42:54.453 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-05T07:42:54.453 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:54.453 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:42:54.453 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================== 2024-02-05T07:42:54.453 INFO:teuthology.orchestra.run.smithi131.stdout:Install 6 Packages 2024-02-05T07:42:54.453 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:54.454 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 6.8 M 2024-02-05T07:42:54.454 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 33 M 2024-02-05T07:42:54.454 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:42:54.602 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:42:54.603 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================== 2024-02-05T07:42:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-02-05T07:42:54.603 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================== 2024-02-05T07:42:54.603 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:42:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 52 k 2024-02-05T07:42:54.603 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-02-05T07:42:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-05T07:42:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-05T07:42:54.603 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-05T07:42:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-05T07:42:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-05T07:42:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-05T07:42:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-05T07:42:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-05T07:42:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-05T07:42:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-05T07:42:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-05T07:42:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-05T07:42:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:54.604 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:42:54.604 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================== 2024-02-05T07:42:54.604 INFO:teuthology.orchestra.run.smithi157.stdout:Install 13 Packages 2024-02-05T07:42:54.604 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:54.605 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 2.0 M 2024-02-05T07:42:54.605 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 26 M 2024-02-05T07:42:54.605 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:42:54.727 INFO:teuthology.orchestra.run.smithi131.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.0 MB/s | 228 kB 00:00 2024-02-05T07:42:54.752 INFO:teuthology.orchestra.run.smithi131.stdout:(2/6): ceph-mgr-cephadm-19.0.0-1241.g164d3be4.e 548 kB/s | 138 kB 00:00 2024-02-05T07:42:54.919 INFO:teuthology.orchestra.run.smithi131.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 5.8 MB 00:00 2024-02-05T07:42:55.086 INFO:teuthology.orchestra.run.smithi131.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 173 kB/s | 58 kB 00:00 2024-02-05T07:42:55.094 INFO:teuthology.orchestra.run.smithi157.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 129 kB/s | 18 kB 00:00 2024-02-05T07:42:55.119 INFO:teuthology.orchestra.run.smithi157.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 147 kB/s | 24 kB 00:00 2024-02-05T07:42:55.178 INFO:teuthology.orchestra.run.smithi157.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 174 kB 00:00 2024-02-05T07:42:55.203 INFO:teuthology.orchestra.run.smithi157.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-02-05T07:42:55.228 INFO:teuthology.orchestra.run.smithi157.stdout:(5/13): ceph-mgr-rook-19.0.0-1241.g164d3be4.el9 190 kB/s | 52 kB 00:00 2024-02-05T07:42:55.254 INFO:teuthology.orchestra.run.smithi157.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 211 kB 00:00 2024-02-05T07:42:55.261 INFO:teuthology.orchestra.run.smithi131.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 149 kB/s | 51 kB 00:00 2024-02-05T07:42:55.279 INFO:teuthology.orchestra.run.smithi157.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 568 kB/s | 43 kB 00:00 2024-02-05T07:42:55.320 INFO:teuthology.orchestra.run.smithi131.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 925 kB/s | 548 kB 00:00 2024-02-05T07:42:55.322 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:42:55.322 INFO:teuthology.orchestra.run.smithi131.stdout:Total 7.8 MB/s | 6.8 MB 00:00 2024-02-05T07:42:55.322 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:42:55.332 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:42:55.332 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:42:55.413 INFO:teuthology.orchestra.run.smithi157.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 112 kB/s | 15 kB 00:00 2024-02-05T07:42:55.426 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:42:55.426 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:42:55.438 INFO:teuthology.orchestra.run.smithi157.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 154 kB/s | 32 kB 00:00 2024-02-05T07:42:55.488 INFO:teuthology.orchestra.run.smithi157.stdout:(10/13): python3-google-auth-2.27.0-1.el9.noarc 971 kB/s | 203 kB 00:00 2024-02-05T07:42:55.521 INFO:teuthology.orchestra.run.smithi157.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 708 kB/s | 59 kB 00:00 2024-02-05T07:42:55.563 INFO:teuthology.orchestra.run.smithi157.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-02-05T07:42:55.580 INFO:teuthology.orchestra.run.smithi157.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 984 kB/s | 90 kB 00:00 2024-02-05T07:42:55.586 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:42:55.586 INFO:teuthology.orchestra.run.smithi157.stdout:Total 2.1 MB/s | 2.0 MB 00:00 2024-02-05T07:42:55.586 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:42:55.599 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:42:55.599 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:42:55.699 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:42:55.700 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:42:55.751 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:42:55.917 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-05T07:42:55.990 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-05T07:42:56.250 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:42:56.430 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-05T07:42:56.482 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-05T07:42:56.500 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-05T07:42:56.537 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-05T07:42:56.574 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-05T07:42:56.583 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-05T07:42:56.616 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-05T07:42:56.675 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2024-02-05T07:42:56.709 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 6/6 2024-02-05T07:42:56.710 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-05T07:42:56.810 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-05T07:42:56.858 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-05T07:42:57.498 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 6/6 2024-02-05T07:42:57.498 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/6 2024-02-05T07:42:57.498 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-05T07:42:57.498 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2024-02-05T07:42:57.498 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-05T07:42:57.499 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-05T07:42:57.587 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-05T07:42:57.639 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-05T07:42:57.672 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-05T07:42:57.721 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-05T07:42:57.742 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:19.0.0-1241.g164d3be4.el9.noarch 13/13 2024-02-05T07:42:57.808 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-05T07:42:57.809 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:57.809 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:42:57.809 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:42:57.809 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-05T07:42:57.809 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-05T07:42:57.809 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-02-05T07:42:57.809 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-05T07:42:57.809 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-05T07:42:57.809 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:57.809 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:42:58.058 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-fuse 2024-02-05T07:42:58.486 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1241.g164d3be4.el9.noarch 13/13 2024-02-05T07:42:58.486 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1241.g164d3be4.el9.noarch 1/13 2024-02-05T07:42:58.486 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-05T07:42:58.486 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-05T07:42:58.486 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-05T07:42:58.486 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-05T07:42:58.486 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-05T07:42:58.486 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-05T07:42:58.486 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-05T07:42:58.486 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-05T07:42:58.486 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-05T07:42:58.486 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-05T07:42:58.487 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-05T07:42:58.625 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:55 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:42:58.784 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:42:58.785 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:42:58.785 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-02-05T07:42:58.785 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:42:58.785 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:42:58.785 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 885 k 2024-02-05T07:42:58.785 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-02-05T07:42:58.785 INFO:teuthology.orchestra.run.smithi131.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-05T07:42:58.785 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:58.785 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:42:58.786 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:42:58.786 INFO:teuthology.orchestra.run.smithi131.stdout:Install 2 Packages 2024-02-05T07:42:58.786 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:42:58.786 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 964 k 2024-02-05T07:42:58.786 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 2.8 M 2024-02-05T07:42:58.786 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:42:58.868 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-05T07:42:58.868 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:58.868 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:42:58.868 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:42:58.868 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-05T07:42:58.868 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-05T07:42:58.869 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-05T07:42:58.869 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-05T07:42:58.869 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-05T07:42:58.869 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-05T07:42:58.869 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-05T07:42:58.869 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-05T07:42:58.869 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-05T07:42:58.869 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-05T07:42:58.869 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-05T07:42:58.869 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-05T07:42:58.869 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:58.869 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:42:58.930 INFO:teuthology.orchestra.run.smithi131.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 548 kB/s | 79 kB 00:00 2024-02-05T07:42:59.151 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr-cephadm 2024-02-05T07:42:59.280 INFO:teuthology.orchestra.run.smithi131.stdout:(2/2): ceph-fuse-19.0.0-1241.g164d3be4.el9.x86_ 1.8 MB/s | 885 kB 00:00 2024-02-05T07:42:59.282 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:42:59.282 INFO:teuthology.orchestra.run.smithi131.stdout:Total 1.9 MB/s | 964 kB 00:00 2024-02-05T07:42:59.282 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:42:59.302 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:42:59.302 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:42:59.378 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:42:59.379 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:42:59.577 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:42:59.715 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-05T07:42:59.718 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:38 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:42:59.779 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 2/2 2024-02-05T07:42:59.875 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:42:59.876 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================== 2024-02-05T07:42:59.876 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-02-05T07:42:59.876 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================== 2024-02-05T07:42:59.876 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:42:59.876 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 138 k 2024-02-05T07:42:59.876 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-02-05T07:42:59.876 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-05T07:42:59.877 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-05T07:42:59.877 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-02-05T07:42:59.877 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-05T07:42:59.877 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-05T07:42:59.877 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:59.877 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:42:59.877 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================== 2024-02-05T07:42:59.877 INFO:teuthology.orchestra.run.smithi157.stdout:Install 6 Packages 2024-02-05T07:42:59.877 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:42:59.878 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 6.8 M 2024-02-05T07:42:59.878 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 33 M 2024-02-05T07:42:59.878 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:43:00.157 INFO:teuthology.orchestra.run.smithi157.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.2 MB/s | 228 kB 00:00 2024-02-05T07:43:00.207 INFO:teuthology.orchestra.run.smithi157.stdout:(2/6): ceph-mgr-cephadm-19.0.0-1241.g164d3be4.e 583 kB/s | 138 kB 00:00 2024-02-05T07:43:00.332 INFO:teuthology.orchestra.run.smithi157.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2024-02-05T07:43:00.358 INFO:teuthology.orchestra.run.smithi157.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 387 kB/s | 58 kB 00:00 2024-02-05T07:43:00.416 INFO:teuthology.orchestra.run.smithi157.stdout:(5/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 2.1 MB/s | 548 kB 00:00 2024-02-05T07:43:00.441 INFO:teuthology.orchestra.run.smithi157.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 470 kB/s | 51 kB 00:00 2024-02-05T07:43:00.443 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:43:00.443 INFO:teuthology.orchestra.run.smithi157.stdout:Total 12 MB/s | 6.8 MB 00:00 2024-02-05T07:43:00.443 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:43:00.452 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:43:00.452 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:43:00.479 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 2/2 2024-02-05T07:43:00.479 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 1/2 2024-02-05T07:43:00.543 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:43:00.543 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:43:00.802 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-05T07:43:00.802 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:00.802 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:43:00.802 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-05T07:43:00.802 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:00.802 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:43:00.839 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:43:01.017 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-05T07:43:01.059 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-05T07:43:01.096 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-volume 2024-02-05T07:43:01.578 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-05T07:43:01.662 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-05T07:43:01.677 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:58 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:43:01.723 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2024-02-05T07:43:01.746 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 6/6 2024-02-05T07:43:01.834 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:43:01.834 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:01.835 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-02-05T07:43:01.835 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:01.835 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:43:01.835 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 263 k 2024-02-05T07:43:01.835 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-02-05T07:43:01.835 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-05T07:43:01.835 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:01.835 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:43:01.835 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:01.835 INFO:teuthology.orchestra.run.smithi131.stdout:Install 2 Packages 2024-02-05T07:43:01.835 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:01.836 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 332 k 2024-02-05T07:43:01.836 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 1.5 M 2024-02-05T07:43:01.836 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:43:01.979 INFO:teuthology.orchestra.run.smithi131.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 481 kB/s | 69 kB 00:00 2024-02-05T07:43:02.229 INFO:teuthology.orchestra.run.smithi131.stdout:(2/2): ceph-volume-19.0.0-1241.g164d3be4.el9.no 670 kB/s | 263 kB 00:00 2024-02-05T07:43:02.230 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:43:02.230 INFO:teuthology.orchestra.run.smithi131.stdout:Total 840 kB/s | 332 kB 00:00 2024-02-05T07:43:02.231 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:43:02.239 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:43:02.239 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:43:02.280 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:43:02.281 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:43:02.430 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:43:02.576 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 6/6 2024-02-05T07:43:02.577 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/6 2024-02-05T07:43:02.577 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-05T07:43:02.577 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2024-02-05T07:43:02.577 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-05T07:43:02.577 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-05T07:43:02.605 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-02-05T07:43:02.640 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-volume-2:19.0.0-1241.g164d3be4.el9.noarch 2/2 2024-02-05T07:43:02.655 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-volume-2:19.0.0-1241.g164d3be4.el9.noarch 2/2 2024-02-05T07:43:02.655 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:43:02.655 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-05T07:43:02.655 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:02.890 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-05T07:43:02.890 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:02.890 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:43:02.890 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:43:02.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-05T07:43:02.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-05T07:43:02.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-02-05T07:43:02.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-05T07:43:02.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-05T07:43:02.891 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:02.891 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:43:03.132 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-fuse 2024-02-05T07:43:03.378 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-volume-2:19.0.0-1241.g164d3be4.el9.noarch 1/2 2024-02-05T07:43:03.703 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:42 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:43:03.733 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-02-05T07:43:03.733 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:03.733 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:43:03.733 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:43:03.734 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-05T07:43:03.734 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:03.734 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:43:03.861 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:43:03.861 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:03.862 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-02-05T07:43:03.862 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:03.862 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:43:03.862 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 885 k 2024-02-05T07:43:03.862 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-02-05T07:43:03.862 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-05T07:43:03.862 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:03.862 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:43:03.862 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:03.862 INFO:teuthology.orchestra.run.smithi157.stdout:Install 2 Packages 2024-02-05T07:43:03.862 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:03.863 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 964 k 2024-02-05T07:43:03.863 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 2.8 M 2024-02-05T07:43:03.863 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:43:04.017 INFO:teuthology.orchestra.run.smithi157.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 510 kB/s | 79 kB 00:00 2024-02-05T07:43:04.085 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install librados-devel 2024-02-05T07:43:04.318 INFO:teuthology.orchestra.run.smithi157.stdout:(2/2): ceph-fuse-19.0.0-1241.g164d3be4.el9.x86_ 1.9 MB/s | 885 kB 00:00 2024-02-05T07:43:04.319 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:43:04.320 INFO:teuthology.orchestra.run.smithi157.stdout:Total 2.1 MB/s | 964 kB 00:00 2024-02-05T07:43:04.320 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:43:04.340 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:43:04.340 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:43:04.417 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:43:04.417 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:43:04.622 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:43:04.664 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:02:01 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:43:04.752 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-05T07:43:04.820 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:43:04.821 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:04.821 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repo Size 2024-02-05T07:43:04.821 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:04.821 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:43:04.822 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 127 k 2024-02-05T07:43:04.822 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:04.822 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:43:04.822 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:04.822 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-02-05T07:43:04.822 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:04.822 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 127 k 2024-02-05T07:43:04.822 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 456 k 2024-02-05T07:43:04.822 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:43:04.825 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 2/2 2024-02-05T07:43:05.044 INFO:teuthology.orchestra.run.smithi131.stdout:librados-devel-19.0.0-1241.g164d3be4.el9.x86_64 573 kB/s | 127 kB 00:00 2024-02-05T07:43:05.044 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:43:05.045 INFO:teuthology.orchestra.run.smithi131.stdout:Total 569 kB/s | 127 kB 00:00 2024-02-05T07:43:05.045 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:43:05.050 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:43:05.050 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:43:05.105 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:43:05.105 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:43:05.255 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:43:05.368 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librados-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:05.525 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 2/2 2024-02-05T07:43:05.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 1/2 2024-02-05T07:43:05.829 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-05T07:43:05.829 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:05.829 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:43:05.830 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-05T07:43:05.830 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:05.830 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:43:05.878 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:06.114 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-volume 2024-02-05T07:43:06.206 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:06.207 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:06.207 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:43:06.207 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:43:06.207 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:06.207 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:43:06.420 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install libcephfs2 2024-02-05T07:43:06.700 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:45 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:43:06.858 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:43:06.859 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:06.859 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-02-05T07:43:06.859 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:06.859 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:43:06.859 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 263 k 2024-02-05T07:43:06.859 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-02-05T07:43:06.859 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-05T07:43:06.859 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:06.859 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:43:06.860 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:06.860 INFO:teuthology.orchestra.run.smithi157.stdout:Install 2 Packages 2024-02-05T07:43:06.860 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:06.860 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 332 k 2024-02-05T07:43:06.860 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 1.5 M 2024-02-05T07:43:06.860 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:43:07.002 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:02:04 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:43:07.005 INFO:teuthology.orchestra.run.smithi157.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 476 kB/s | 69 kB 00:00 2024-02-05T07:43:07.110 INFO:teuthology.orchestra.run.smithi131.stdout:Package libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:43:07.122 INFO:teuthology.orchestra.run.smithi157.stdout:(2/2): ceph-volume-19.0.0-1241.g164d3be4.el9.no 1.0 MB/s | 263 kB 00:00 2024-02-05T07:43:07.124 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:43:07.124 INFO:teuthology.orchestra.run.smithi157.stdout:Total 1.2 MB/s | 332 kB 00:00 2024-02-05T07:43:07.124 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:43:07.132 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:43:07.132 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:43:07.162 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:43:07.164 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-02-05T07:43:07.164 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:43:07.172 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:43:07.172 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:43:07.219 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install libcephfs-devel 2024-02-05T07:43:07.322 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:43:07.498 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-02-05T07:43:07.541 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:19.0.0-1241.g164d3be4.el9.noarch 2/2 2024-02-05T07:43:07.557 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:19.0.0-1241.g164d3be4.el9.noarch 2/2 2024-02-05T07:43:07.558 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:43:07.558 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-05T07:43:07.558 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:07.784 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:02:04 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:43:07.941 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:43:07.942 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repo Size 2024-02-05T07:43:07.943 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:07.943 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:43:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 31 k 2024-02-05T07:43:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:07.943 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:43:07.943 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:07.943 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-02-05T07:43:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:07.943 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 31 k 2024-02-05T07:43:07.943 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 140 k 2024-02-05T07:43:07.943 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:43:08.182 INFO:teuthology.orchestra.run.smithi131.stdout:libcephfs-devel-19.0.0-1241.g164d3be4.el9.x86_6 131 kB/s | 31 kB 00:00 2024-02-05T07:43:08.182 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:43:08.182 INFO:teuthology.orchestra.run.smithi131.stdout:Total 130 kB/s | 31 kB 00:00 2024-02-05T07:43:08.182 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:43:08.185 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:43:08.185 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:43:08.214 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:43:08.214 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:43:08.294 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:19.0.0-1241.g164d3be4.el9.noarch 1/2 2024-02-05T07:43:08.306 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:43:08.429 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:08.647 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-02-05T07:43:08.647 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:08.647 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:43:08.647 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:43:08.647 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-05T07:43:08.647 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:08.648 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:43:08.853 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:08.979 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install librados-devel 2024-02-05T07:43:09.132 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:09.132 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:09.132 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:43:09.132 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:43:09.132 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:09.133 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:43:09.311 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install librados2 2024-02-05T07:43:09.555 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:48 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:43:09.713 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:43:09.714 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:09.714 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repo Size 2024-02-05T07:43:09.714 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:09.714 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:43:09.714 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 127 k 2024-02-05T07:43:09.714 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:09.714 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:43:09.714 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:09.714 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-02-05T07:43:09.714 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:09.715 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 127 k 2024-02-05T07:43:09.715 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 456 k 2024-02-05T07:43:09.715 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:43:09.884 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:02:06 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:43:09.953 INFO:teuthology.orchestra.run.smithi157.stdout:librados-devel-19.0.0-1241.g164d3be4.el9.x86_64 533 kB/s | 127 kB 00:00 2024-02-05T07:43:09.953 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:43:09.954 INFO:teuthology.orchestra.run.smithi157.stdout:Total 530 kB/s | 127 kB 00:00 2024-02-05T07:43:09.954 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:43:09.959 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:43:09.959 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:43:09.987 INFO:teuthology.orchestra.run.smithi131.stdout:Package librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:43:10.016 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:43:10.016 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:43:10.039 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:43:10.041 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-02-05T07:43:10.041 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:43:10.101 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install librbd1 2024-02-05T07:43:10.163 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:43:10.259 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:10.667 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:02:07 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:43:10.703 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:10.771 INFO:teuthology.orchestra.run.smithi131.stdout:Package librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:43:10.823 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:43:10.825 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-02-05T07:43:10.825 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:43:10.880 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install python3-rados 2024-02-05T07:43:11.020 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:11.020 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:11.020 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:43:11.020 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:43:11.020 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:11.020 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:43:11.299 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install libcephfs2 2024-02-05T07:43:11.450 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:02:08 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:43:11.554 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-rados-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:43:11.606 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:43:11.607 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-02-05T07:43:11.607 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:43:11.665 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install python3-rgw 2024-02-05T07:43:11.889 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:50 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:43:11.993 INFO:teuthology.orchestra.run.smithi157.stdout:Package libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:43:12.045 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:43:12.046 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-02-05T07:43:12.046 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:43:12.104 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install libcephfs-devel 2024-02-05T07:43:12.236 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:02:09 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:43:12.339 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-rgw-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:43:12.393 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:43:12.394 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-02-05T07:43:12.394 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:43:12.450 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install python3-cephfs 2024-02-05T07:43:12.668 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:51 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:43:12.823 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:43:12.824 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:12.824 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repo Size 2024-02-05T07:43:12.825 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:12.825 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:43:12.825 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 31 k 2024-02-05T07:43:12.825 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:12.825 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:43:12.825 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:12.825 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-02-05T07:43:12.825 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:12.825 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 31 k 2024-02-05T07:43:12.825 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 140 k 2024-02-05T07:43:12.825 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:43:12.983 INFO:teuthology.orchestra.run.smithi157.stdout:libcephfs-devel-19.0.0-1241.g164d3be4.el9.x86_6 198 kB/s | 31 kB 00:00 2024-02-05T07:43:12.984 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:43:12.984 INFO:teuthology.orchestra.run.smithi157.stdout:Total 196 kB/s | 31 kB 00:00 2024-02-05T07:43:12.984 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:43:12.987 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:43:12.987 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:43:13.016 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:02:10 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:43:13.018 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:43:13.018 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:43:13.111 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:43:13.120 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-cephfs-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:43:13.172 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:43:13.173 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-02-05T07:43:13.173 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:43:13.213 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:13.229 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install python3-rbd 2024-02-05T07:43:13.633 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:13.808 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:02:10 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:43:13.895 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:13.895 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:13.895 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:43:13.895 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:43:13.895 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:13.895 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:43:13.912 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-rbd-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:43:13.964 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:43:13.966 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-02-05T07:43:13.966 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:43:14.022 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install rbd-fuse 2024-02-05T07:43:14.080 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install librados2 2024-02-05T07:43:14.590 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:02:11 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:43:14.665 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:53 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:43:14.748 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:43:14.748 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:14.748 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-02-05T07:43:14.748 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:14.748 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:43:14.748 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 85 k 2024-02-05T07:43:14.749 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:14.749 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:43:14.749 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:14.749 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-02-05T07:43:14.749 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:14.749 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 85 k 2024-02-05T07:43:14.749 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 231 k 2024-02-05T07:43:14.750 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:43:14.776 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:43:14.829 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:43:14.831 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-02-05T07:43:14.831 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:43:14.914 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install librbd1 2024-02-05T07:43:14.951 INFO:teuthology.orchestra.run.smithi131.stdout:rbd-fuse-19.0.0-1241.g164d3be4.el9.x86_64.rpm 424 kB/s | 85 kB 00:00 2024-02-05T07:43:14.951 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:43:14.951 INFO:teuthology.orchestra.run.smithi131.stdout:Total 422 kB/s | 85 kB 00:00 2024-02-05T07:43:14.952 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:43:14.959 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:43:14.959 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:43:15.014 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:43:15.015 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:43:15.182 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:43:15.283 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:15.510 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:54 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:43:15.623 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:43:15.676 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:43:15.677 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-02-05T07:43:15.677 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:43:15.702 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:15.735 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install python3-rados 2024-02-05T07:43:16.011 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:16.012 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:16.012 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:43:16.012 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:43:16.012 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:16.012 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:43:16.240 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install rbd-mirror 2024-02-05T07:43:16.303 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:55 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:43:16.407 INFO:teuthology.orchestra.run.smithi157.stdout:Package python3-rados-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:43:16.458 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:43:16.459 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-02-05T07:43:16.460 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:43:16.516 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install python3-rgw 2024-02-05T07:43:16.836 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:02:13 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:43:16.998 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:43:16.999 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:16.999 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repo Size 2024-02-05T07:43:16.999 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:16.999 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:43:16.999 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 3.1 M 2024-02-05T07:43:16.999 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:16.999 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:43:16.999 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:16.999 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-02-05T07:43:16.999 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:16.999 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 3.1 M 2024-02-05T07:43:16.999 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 13 M 2024-02-05T07:43:16.999 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:43:17.083 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:56 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:43:17.188 INFO:teuthology.orchestra.run.smithi157.stdout:Package python3-rgw-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:43:17.240 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:43:17.241 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-02-05T07:43:17.241 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:43:17.306 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install python3-cephfs 2024-02-05T07:43:17.605 INFO:teuthology.orchestra.run.smithi131.stdout:rbd-mirror-19.0.0-1241.g164d3be4.el9.x86_64.rpm 5.1 MB/s | 3.1 MB 00:00 2024-02-05T07:43:17.606 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:43:17.606 INFO:teuthology.orchestra.run.smithi131.stdout:Total 5.1 MB/s | 3.1 MB 00:00 2024-02-05T07:43:17.606 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:43:17.615 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:43:17.615 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:43:17.689 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:43:17.689 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:43:17.867 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:56 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:43:17.972 INFO:teuthology.orchestra.run.smithi157.stdout:Package python3-cephfs-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:43:18.024 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:43:18.025 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-02-05T07:43:18.025 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:43:18.088 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install python3-rbd 2024-02-05T07:43:18.112 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:43:18.181 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-mirror-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:18.206 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:18.206 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:43:18.206 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-05T07:43:18.206 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-05T07:43:18.206 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-05T07:43:18.206 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:18.658 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:57 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:43:18.762 INFO:teuthology.orchestra.run.smithi157.stdout:Package python3-rbd-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:43:18.814 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:43:18.815 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-02-05T07:43:18.815 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:43:18.877 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install rbd-fuse 2024-02-05T07:43:19.443 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:58 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:43:19.464 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-mirror-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:19.464 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:19.464 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:43:19.464 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:43:19.464 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:19.465 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:43:19.599 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:43:19.600 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:19.600 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-02-05T07:43:19.600 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:19.600 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:43:19.600 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 85 k 2024-02-05T07:43:19.600 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:19.600 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:43:19.600 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:19.600 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-02-05T07:43:19.600 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:19.601 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 85 k 2024-02-05T07:43:19.601 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 231 k 2024-02-05T07:43:19.601 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:43:19.720 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install rbd-nbd 2024-02-05T07:43:19.911 INFO:teuthology.orchestra.run.smithi157.stdout:rbd-fuse-19.0.0-1241.g164d3be4.el9.x86_64.rpm 275 kB/s | 85 kB 00:00 2024-02-05T07:43:19.912 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:43:19.912 INFO:teuthology.orchestra.run.smithi157.stdout:Total 274 kB/s | 85 kB 00:00 2024-02-05T07:43:19.912 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:43:19.919 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:43:19.919 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:43:19.975 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:43:19.975 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:43:20.148 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:43:20.252 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:20.302 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:02:17 ago on Mon 05 Feb 2024 07:41:03 AM UTC. 2024-02-05T07:43:20.458 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-02-05T07:43:20.459 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:20.459 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-02-05T07:43:20.459 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:20.459 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-02-05T07:43:20.459 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 171 k 2024-02-05T07:43:20.459 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:20.459 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-02-05T07:43:20.459 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-02-05T07:43:20.459 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-02-05T07:43:20.459 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:20.460 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 171 k 2024-02-05T07:43:20.460 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 494 k 2024-02-05T07:43:20.460 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-02-05T07:43:20.683 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:20.809 INFO:teuthology.orchestra.run.smithi131.stdout:rbd-nbd-19.0.0-1241.g164d3be4.el9.x86_64.rpm 490 kB/s | 171 kB 00:00 2024-02-05T07:43:20.809 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:43:20.810 INFO:teuthology.orchestra.run.smithi131.stdout:Total 488 kB/s | 171 kB 00:00 2024-02-05T07:43:20.810 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-02-05T07:43:20.818 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-02-05T07:43:20.818 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-02-05T07:43:20.874 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-02-05T07:43:20.875 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-02-05T07:43:20.986 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:20.986 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:20.986 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:43:20.986 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:43:20.986 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:20.986 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:43:21.028 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-02-05T07:43:21.132 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-nbd-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:21.238 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install rbd-mirror 2024-02-05T07:43:21.562 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-nbd-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:21.828 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:02:00 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:43:21.871 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-nbd-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:21.871 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:21.871 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-02-05T07:43:21.871 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:43:21.871 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:43:21.871 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-02-05T07:43:21.983 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:43:21.984 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:21.984 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repo Size 2024-02-05T07:43:21.984 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:21.984 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:43:21.984 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 3.1 M 2024-02-05T07:43:21.984 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:21.984 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:43:21.984 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:21.984 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-02-05T07:43:21.984 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:21.985 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 3.1 M 2024-02-05T07:43:21.985 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 13 M 2024-02-05T07:43:21.985 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:43:22.096 DEBUG:teuthology.parallel:result is None 2024-02-05T07:43:22.580 INFO:teuthology.orchestra.run.smithi157.stdout:rbd-mirror-19.0.0-1241.g164d3be4.el9.x86_64.rpm 5.2 MB/s | 3.1 MB 00:00 2024-02-05T07:43:22.581 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:43:22.581 INFO:teuthology.orchestra.run.smithi157.stdout:Total 5.2 MB/s | 3.1 MB 00:00 2024-02-05T07:43:22.581 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:43:22.591 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:43:22.591 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:43:22.665 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:43:22.666 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:43:23.096 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:43:23.165 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:23.190 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:23.190 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:43:23.190 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-05T07:43:23.190 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-05T07:43:23.190 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-05T07:43:23.190 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:24.369 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:24.369 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:24.369 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:43:24.369 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:43:24.369 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:24.370 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:43:24.616 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install rbd-nbd 2024-02-05T07:43:25.207 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:02:04 ago on Mon 05 Feb 2024 07:41:21 AM UTC. 2024-02-05T07:43:25.365 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-02-05T07:43:25.366 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:25.366 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-02-05T07:43:25.366 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:25.366 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-02-05T07:43:25.366 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 171 k 2024-02-05T07:43:25.366 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:25.366 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-02-05T07:43:25.366 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-02-05T07:43:25.366 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-02-05T07:43:25.366 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:25.366 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 171 k 2024-02-05T07:43:25.366 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 494 k 2024-02-05T07:43:25.366 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-02-05T07:43:25.592 INFO:teuthology.orchestra.run.smithi157.stdout:rbd-nbd-19.0.0-1241.g164d3be4.el9.x86_64.rpm 760 kB/s | 171 kB 00:00 2024-02-05T07:43:25.592 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:43:25.593 INFO:teuthology.orchestra.run.smithi157.stdout:Total 753 kB/s | 171 kB 00:00 2024-02-05T07:43:25.593 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-02-05T07:43:25.601 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-02-05T07:43:25.603 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-02-05T07:43:25.657 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-02-05T07:43:25.658 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-02-05T07:43:25.801 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-02-05T07:43:25.907 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:26.371 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-nbd-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:26.676 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:43:26.677 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:26.677 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-02-05T07:43:26.677 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:43:26.677 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:43:26.677 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-02-05T07:43:26.887 DEBUG:teuthology.parallel:result is None 2024-02-05T07:43:26.887 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:43:27.028 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-05T07:43:27.052 INFO:teuthology.orchestra.run.smithi131.stdout:19.0.0-1241.g164d3be4.el9 2024-02-05T07:43:27.053 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1241.g164d3be4.el9 2024-02-05T07:43:27.053 INFO:teuthology.task.install:The correct ceph version 19.0.0-1241.g164d3be4 is installed. 2024-02-05T07:43:27.054 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:43:27.204 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-05T07:43:27.226 INFO:teuthology.orchestra.run.smithi157.stdout:19.0.0-1241.g164d3be4.el9 2024-02-05T07:43:27.226 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1241.g164d3be4.el9 2024-02-05T07:43:27.226 INFO:teuthology.task.install:The correct ceph version 19.0.0-1241.g164d3be4 is installed. 2024-02-05T07:43:27.227 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-05T07:43:27.228 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:43:27.228 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-05T07:43:27.261 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-02-05T07:43:27.261 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-05T07:43:27.294 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-05T07:43:27.294 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:43:27.294 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2024-02-05T07:43:27.331 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-05T07:43:27.400 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-02-05T07:43:27.400 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-02-05T07:43:27.428 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-05T07:43:27.495 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-05T07:43:27.495 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:43:27.495 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-05T07:43:27.524 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-05T07:43:27.593 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-02-05T07:43:27.593 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-05T07:43:27.620 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-05T07:43:27.687 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-05T07:43:27.687 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:43:27.687 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2024-02-05T07:43:27.715 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-05T07:43:27.783 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-02-05T07:43:27.783 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-02-05T07:43:27.811 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-05T07:43:27.877 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-05T07:43:27.967 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '164d3be4a527be775f4d3c343ea591122f1b8921', 'cephadm_mode': 'cephadm-package'} 2024-02-05T07:43:27.968 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:43:27.968 INFO:tasks.cephadm:Cluster fsid is 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:43:27.968 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-05T07:43:27.968 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.131', 'mon.c': '[v2:172.21.15.131:3301,v1:172.21.15.131:6790]', 'mon.b': '172.21.15.157'} 2024-02-05T07:43:27.968 INFO:tasks.cephadm:First mon is mon.a on smithi131 2024-02-05T07:43:27.968 INFO:tasks.cephadm:First mgr is y 2024-02-05T07:43:27.969 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-05T07:43:27.969 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname $(hostname -s) 2024-02-05T07:43:27.997 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname $(hostname -s) 2024-02-05T07:43:28.033 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 on all hosts... 2024-02-05T07:43:28.033 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 pull 2024-02-05T07:43:28.040 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 pull 2024-02-05T07:43:28.863 INFO:teuthology.orchestra.run.smithi131.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921... 2024-02-05T07:43:28.885 INFO:teuthology.orchestra.run.smithi157.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921... 2024-02-05T07:43:56.911 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-02-05T07:43:56.912 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev)", 2024-02-05T07:43:56.912 INFO:teuthology.orchestra.run.smithi131.stdout: "image_id": "8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1", 2024-02-05T07:43:56.912 INFO:teuthology.orchestra.run.smithi131.stdout: "repo_digests": [ 2024-02-05T07:43:56.912 INFO:teuthology.orchestra.run.smithi131.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0" 2024-02-05T07:43:56.912 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-02-05T07:43:56.912 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-02-05T07:43:58.151 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-02-05T07:43:58.151 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev)", 2024-02-05T07:43:58.151 INFO:teuthology.orchestra.run.smithi157.stdout: "image_id": "8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1", 2024-02-05T07:43:58.151 INFO:teuthology.orchestra.run.smithi157.stdout: "repo_digests": [ 2024-02-05T07:43:58.152 INFO:teuthology.orchestra.run.smithi157.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0" 2024-02-05T07:43:58.152 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2024-02-05T07:43:58.152 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-02-05T07:43:58.175 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph 2024-02-05T07:43:58.210 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph 2024-02-05T07:43:58.240 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /etc/ceph 2024-02-05T07:43:58.280 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /etc/ceph 2024-02-05T07:43:58.307 INFO:tasks.cephadm:Writing seed config... 2024-02-05T07:43:58.308 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-05T07:43:58.308 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-05T07:43:58.308 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-05T07:43:58.308 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-02-05T07:43:58.309 INFO:tasks.cephadm: override: [global] ms bind msgr1 = False 2024-02-05T07:43:58.309 INFO:tasks.cephadm: override: [global] ms bind msgr2 = True 2024-02-05T07:43:58.309 INFO:tasks.cephadm: override: [global] ms type = async 2024-02-05T07:43:58.309 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-05T07:43:58.309 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-05T07:43:58.309 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-05T07:43:58.309 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-02-05T07:43:58.309 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-05T07:43:58.309 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-05T07:43:58.309 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-02-05T07:43:58.309 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-02-05T07:43:58.309 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-05T07:43:58.310 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:43:58.310 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-05T07:43:58.336 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 4018fbe8-c3fa-11ee-95b6-87774f69a715 mon election default strategy = 1 ms bind msgr1 = False ms bind msgr2 = True ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-02-05T07:43:58.336 DEBUG:teuthology.orchestra.run.smithi131:mon.a> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mon.a.service 2024-02-05T07:43:58.379 DEBUG:teuthology.orchestra.run.smithi131:mgr.y> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mgr.y.service 2024-02-05T07:43:58.421 INFO:tasks.cephadm:Bootstrapping... 2024-02-05T07:43:58.421 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 -v bootstrap --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.131 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-05T07:43:58.613 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:43:58.613 INFO:teuthology.orchestra.run.smithi131.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921', '-v', 'bootstrap', '--fsid', '4018fbe8-c3fa-11ee-95b6-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.131', '--skip-admin-label'] 2024-02-05T07:43:58.639 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 4.9.0 2024-02-05T07:43:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-05T07:43:58.640 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying podman|docker is present... 2024-02-05T07:43:58.664 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 4.9.0 2024-02-05T07:43:58.664 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying lvm2 is present... 2024-02-05T07:43:58.664 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying time synchronization is in place... 2024-02-05T07:43:58.671 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-05T07:43:58.671 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-05T07:43:58.677 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-05T07:43:58.677 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-02-05T07:43:58.683 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout enabled 2024-02-05T07:43:58.689 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout active 2024-02-05T07:43:58.689 INFO:teuthology.orchestra.run.smithi131.stdout:Unit chronyd.service is enabled and running 2024-02-05T07:43:58.689 INFO:teuthology.orchestra.run.smithi131.stdout:Repeating the final host check... 2024-02-05T07:43:58.712 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 4.9.0 2024-02-05T07:43:58.712 INFO:teuthology.orchestra.run.smithi131.stdout:podman (/bin/podman) version 4.9.0 is present 2024-02-05T07:43:58.712 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl is present 2024-02-05T07:43:58.712 INFO:teuthology.orchestra.run.smithi131.stdout:lvcreate is present 2024-02-05T07:43:58.718 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-05T07:43:58.718 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-05T07:43:58.724 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-05T07:43:58.724 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-02-05T07:43:58.730 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout enabled 2024-02-05T07:43:58.737 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout active 2024-02-05T07:43:58.737 INFO:teuthology.orchestra.run.smithi131.stdout:Unit chronyd.service is enabled and running 2024-02-05T07:43:58.737 INFO:teuthology.orchestra.run.smithi131.stdout:Host looks OK 2024-02-05T07:43:58.737 INFO:teuthology.orchestra.run.smithi131.stdout:Cluster fsid: 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:43:58.737 INFO:teuthology.orchestra.run.smithi131.stdout:Acquiring lock 140341155467856 on /run/cephadm/4018fbe8-c3fa-11ee-95b6-87774f69a715.lock 2024-02-05T07:43:58.737 INFO:teuthology.orchestra.run.smithi131.stdout:Lock 140341155467856 acquired on /run/cephadm/4018fbe8-c3fa-11ee-95b6-87774f69a715.lock 2024-02-05T07:43:58.737 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying IP 172.21.15.131 port 3300 ... 2024-02-05T07:43:58.738 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying IP 172.21.15.131 port 6789 ... 2024-02-05T07:43:58.738 INFO:teuthology.orchestra.run.smithi131.stdout:Base mon IP(s) is [172.21.15.131:3300, 172.21.15.131:6789], mon addrv is [v2:172.21.15.131:3300,v1:172.21.15.131:6789] 2024-02-05T07:43:58.740 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.131 metric 100 2024-02-05T07:43:58.740 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.131 metric 100 2024-02-05T07:43:58.743 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-05T07:43:58.743 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-05T07:43:58.743 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1764sec hoplimit 64 pref medium 2024-02-05T07:43:58.745 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-05T07:43:58.745 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-05T07:43:58.745 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-05T07:43:58.745 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-05T07:43:58.745 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-02-05T07:43:58.745 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-05T07:43:58.746 INFO:teuthology.orchestra.run.smithi131.stdout:Mon IP `172.21.15.131` is in CIDR network `172.21.0.0/20` 2024-02-05T07:43:58.746 INFO:teuthology.orchestra.run.smithi131.stdout:Mon IP `172.21.15.131` is in CIDR network `172.21.0.0/20` 2024-02-05T07:43:58.746 INFO:teuthology.orchestra.run.smithi131.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-05T07:43:58.746 INFO:teuthology.orchestra.run.smithi131.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-05T07:43:58.747 INFO:teuthology.orchestra.run.smithi131.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921... 2024-02-05T07:43:59.387 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 2024-02-05T07:43:59.388 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921... 2024-02-05T07:43:59.388 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Getting image source signatures 2024-02-05T07:43:59.388 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying blob sha256:58df4e03ba066accf3a7e07426bab5f1c867e27a139801c642bdafb42b9a2d9a 2024-02-05T07:43:59.388 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying blob sha256:48ac04de0ee67de328b3ea21477b81735e6e7c0783ebcda067ca4695a8266750 2024-02-05T07:43:59.388 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying config sha256:8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 2024-02-05T07:43:59.388 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-05T07:44:00.367 INFO:teuthology.orchestra.run.smithi131.stdout:ceph: stdout ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev) 2024-02-05T07:44:00.367 INFO:teuthology.orchestra.run.smithi131.stdout:Ceph version: ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev) 2024-02-05T07:44:00.367 INFO:teuthology.orchestra.run.smithi131.stdout:Extracting ceph user uid/gid from container image... 2024-02-05T07:44:01.448 INFO:teuthology.orchestra.run.smithi131.stdout:stat: stdout 167 167 2024-02-05T07:44:01.448 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial keys... 2024-02-05T07:44:02.395 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQDBkcBl6nc8MhAA/nfrfvjwJF2GmXmfVJgMdA== 2024-02-05T07:44:03.327 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQDCkcBlbek3LhAArgiA3LdKR3cr8pfhid+8zw== 2024-02-05T07:44:04.416 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQDDkcBlXPEAKRAA4Av7p+0StkpXfIw8+qRO1Q== 2024-02-05T07:44:04.416 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial monmap... 2024-02-05T07:44:05.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-05T07:44:05.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-05T07:44:05.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:44:05.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-05T07:44:05.330 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool for a [v2:172.21.15.131:3300,v1:172.21.15.131:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-05T07:44:05.330 INFO:teuthology.orchestra.run.smithi131.stdout:setting min_mon_release = quincy 2024-02-05T07:44:05.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: set fsid to 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:44:05.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-05T07:44:05.330 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:44:05.330 INFO:teuthology.orchestra.run.smithi131.stdout:Creating mon... 2024-02-05T07:44:06.219 INFO:teuthology.orchestra.run.smithi131.stdout:create mon.a on 2024-02-05T07:44:06.414 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-05T07:44:06.551 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-05T07:44:06.745 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715.target → /etc/systemd/system/ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715.target. 2024-02-05T07:44:06.745 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715.target → /etc/systemd/system/ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715.target. 2024-02-05T07:44:06.969 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mon.a 2024-02-05T07:44:06.969 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mon.a.service: Unit ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mon.a.service not loaded. 2024-02-05T07:44:07.129 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715.target.wants/ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mon.a.service → /etc/systemd/system/ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@.service. 2024-02-05T07:44:07.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:07 smithi131 podman[28981]: 2024-02-05 07:44:07.376582547 +0000 UTC m=+0.127679280 container create 72f3a427dabef48bbd524370cc33a8bd863be84f1729e8057123e447515b7989 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=) 2024-02-05T07:44:07.708 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-05T07:44:07.708 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-02-05T07:44:07.715 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-05T07:44:07.715 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-02-05T07:44:07.715 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-02-05T07:44:07.715 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to enable service . firewalld.service is not available 2024-02-05T07:44:07.715 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mon to start... 2024-02-05T07:44:07.715 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mon... 2024-02-05T07:44:07.838 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:07 smithi131 podman[28981]: 2024-02-05 07:44:07.587225645 +0000 UTC m=+0.338322339 container init 72f3a427dabef48bbd524370cc33a8bd863be84f1729e8057123e447515b7989 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-05T07:44:07.838 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:07 smithi131 podman[28981]: 2024-02-05 07:44:07.592760199 +0000 UTC m=+0.343856890 container start 72f3a427dabef48bbd524370cc33a8bd863be84f1729e8057123e447515b7989 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD) 2024-02-05T07:44:07.838 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:07 smithi131 ceph-mon[29019]: mkfs 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:44:07.838 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:07 smithi131 ceph-mon[29019]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-05T07:44:07.839 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:07 smithi131 bash[28981]: 72f3a427dabef48bbd524370cc33a8bd863be84f1729e8057123e447515b7989 2024-02-05T07:44:07.839 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:07 smithi131 systemd[1]: Started Ceph mon.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T07:44:08.860 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:08 smithi131 ceph-mon[29019]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-05T07:44:08.860 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:08 smithi131 ceph-mon[29019]: monmap e1: 1 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T07:44:08.860 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:08 smithi131 ceph-mon[29019]: fsmap 2024-02-05T07:44:08.860 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:08 smithi131 ceph-mon[29019]: osdmap e1: 0 total, 0 up, 0 in 2024-02-05T07:44:08.860 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:08 smithi131 ceph-mon[29019]: mgrmap e1: no daemons active 2024-02-05T07:44:08.860 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:08 smithi131 ceph-mon[29019]: from='client.? 172.21.15.131:0/2925515368' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-05T07:44:08.862 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout cluster: 2024-02-05T07:44:08.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout id: 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:44:08.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-05T07:44:08.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-02-05T07:44:08.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout services: 2024-02-05T07:44:08.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.57406s) 2024-02-05T07:44:08.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-05T07:44:08.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-05T07:44:08.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-02-05T07:44:08.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout data: 2024-02-05T07:44:08.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-05T07:44:08.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-05T07:44:08.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-05T07:44:08.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout pgs: 2024-02-05T07:44:08.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-02-05T07:44:08.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.156+0000 7fc819106700 1 Processor -- start 2024-02-05T07:44:08.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.156+0000 7fc819106700 1 -- start start 2024-02-05T07:44:08.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.156+0000 7fc819106700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc814105e60 0x7fc814106280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:08.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.156+0000 7fc819106700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141067c0 con 0x7fc814105e60 2024-02-05T07:44:08.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.156+0000 7fc812d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc814105e60 0x7fc814106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:08.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.156+0000 7fc812d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc814105e60 0x7fc814106280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38212/0 (socket says 172.21.15.131:38212) 2024-02-05T07:44:08.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.156+0000 7fc812d9d700 1 -- 172.21.15.131:0/3653448887 learned_addr learned my addr 172.21.15.131:0/3653448887 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:08.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.157+0000 7fc812d9d700 1 -- 172.21.15.131:0/3653448887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc814106900 con 0x7fc814105e60 2024-02-05T07:44:08.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.157+0000 7fc812d9d700 1 --2- 172.21.15.131:0/3653448887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc814105e60 0x7fc814106280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009e80 tx=0x7fc80c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452ca0be5528fe79 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:08.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.157+0000 7fc811d9b700 1 -- 172.21.15.131:0/3653448887 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc80c00e070 con 0x7fc814105e60 2024-02-05T07:44:08.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.157+0000 7fc811d9b700 1 -- 172.21.15.131:0/3653448887 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80c003a20 con 0x7fc814105e60 2024-02-05T07:44:08.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.157+0000 7fc811d9b700 1 -- 172.21.15.131:0/3653448887 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc80c003ce0 con 0x7fc814105e60 2024-02-05T07:44:08.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.157+0000 7fc819106700 1 -- 172.21.15.131:0/3653448887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc814105e60 msgr2=0x7fc814106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:08.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.157+0000 7fc819106700 1 --2- 172.21.15.131:0/3653448887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc814105e60 0x7fc814106280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009e80 tx=0x7fc80c00b330 comp rx=0 tx=0).stop 2024-02-05T07:44:08.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.158+0000 7fc819106700 1 -- 172.21.15.131:0/3653448887 shutdown_connections 2024-02-05T07:44:08.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.158+0000 7fc819106700 1 --2- 172.21.15.131:0/3653448887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc814105e60 0x7fc814106280 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:08.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.158+0000 7fc819106700 1 -- 172.21.15.131:0/3653448887 >> 172.21.15.131:0/3653448887 conn(0x7fc8141014d0 msgr2=0x7fc8141038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:08.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.158+0000 7fc819106700 1 -- 172.21.15.131:0/3653448887 shutdown_connections 2024-02-05T07:44:08.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.158+0000 7fc819106700 1 -- 172.21.15.131:0/3653448887 wait complete. 2024-02-05T07:44:08.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.158+0000 7fc819106700 1 Processor -- start 2024-02-05T07:44:08.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.159+0000 7fc819106700 1 -- start start 2024-02-05T07:44:08.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.159+0000 7fc819106700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc814105e60 0x7fc8141992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:08.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.159+0000 7fc819106700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141997e0 con 0x7fc814105e60 2024-02-05T07:44:08.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.159+0000 7fc812d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc814105e60 0x7fc8141992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:08.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.159+0000 7fc812d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc814105e60 0x7fc8141992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38222/0 (socket says 172.21.15.131:38222) 2024-02-05T07:44:08.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.159+0000 7fc812d9d700 1 -- 172.21.15.131:0/2925515368 learned_addr learned my addr 172.21.15.131:0/2925515368 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:08.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.159+0000 7fc812d9d700 1 -- 172.21.15.131:0/2925515368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80c009b30 con 0x7fc814105e60 2024-02-05T07:44:08.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.159+0000 7fc812d9d700 1 --2- 172.21.15.131:0/2925515368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc814105e60 0x7fc8141992a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc80c004000 tx=0x7fc80c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:08.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.159+0000 7fc80bfff700 1 -- 172.21.15.131:0/2925515368 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc80c004370 con 0x7fc814105e60 2024-02-05T07:44:08.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.159+0000 7fc80bfff700 1 -- 172.21.15.131:0/2925515368 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80c004510 con 0x7fc814105e60 2024-02-05T07:44:08.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.159+0000 7fc819106700 1 -- 172.21.15.131:0/2925515368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8141999e0 con 0x7fc814105e60 2024-02-05T07:44:08.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.160+0000 7fc80bfff700 1 -- 172.21.15.131:0/2925515368 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc80c0047f0 con 0x7fc814105e60 2024-02-05T07:44:08.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.160+0000 7fc819106700 1 -- 172.21.15.131:0/2925515368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc814199e80 con 0x7fc814105e60 2024-02-05T07:44:08.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.160+0000 7fc80bfff700 1 -- 172.21.15.131:0/2925515368 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc80c0127c0 con 0x7fc814105e60 2024-02-05T07:44:08.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.160+0000 7fc80bfff700 1 -- 172.21.15.131:0/2925515368 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc80c026410 con 0x7fc814105e60 2024-02-05T07:44:08.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.160+0000 7fc819106700 1 -- 172.21.15.131:0/2925515368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc814192f60 con 0x7fc814105e60 2024-02-05T07:44:08.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.163+0000 7fc80bfff700 1 -- 172.21.15.131:0/2925515368 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc80c02b060 con 0x7fc814105e60 2024-02-05T07:44:08.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.232+0000 7fc819106700 1 -- 172.21.15.131:0/2925515368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fc8141071d0 con 0x7fc814105e60 2024-02-05T07:44:08.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.232+0000 7fc80bfff700 1 -- 172.21.15.131:0/2925515368 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+316 (secure 0 0 0) 0x7fc80c021030 con 0x7fc814105e60 2024-02-05T07:44:08.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.233+0000 7fc819106700 1 -- 172.21.15.131:0/2925515368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc814105e60 msgr2=0x7fc8141992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:08.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.233+0000 7fc819106700 1 --2- 172.21.15.131:0/2925515368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc814105e60 0x7fc8141992a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc80c004000 tx=0x7fc80c004ad0 comp rx=0 tx=0).stop 2024-02-05T07:44:08.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.233+0000 7fc819106700 1 -- 172.21.15.131:0/2925515368 shutdown_connections 2024-02-05T07:44:08.873 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.233+0000 7fc819106700 1 --2- 172.21.15.131:0/2925515368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc814105e60 0x7fc8141992a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:08.873 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.233+0000 7fc819106700 1 -- 172.21.15.131:0/2925515368 >> 172.21.15.131:0/2925515368 conn(0x7fc8141014d0 msgr2=0x7fc8141900a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:08.873 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.233+0000 7fc819106700 1 -- 172.21.15.131:0/2925515368 shutdown_connections 2024-02-05T07:44:08.873 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:08.233+0000 7fc819106700 1 -- 172.21.15.131:0/2925515368 wait complete. 2024-02-05T07:44:08.873 INFO:teuthology.orchestra.run.smithi131.stdout:mon is available 2024-02-05T07:44:08.873 INFO:teuthology.orchestra.run.smithi131.stdout:Assimilating anything we can from ceph.conf... 2024-02-05T07:44:09.933 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:09 smithi131 ceph-mon[29019]: from='client.? 172.21.15.131:0/3026303119' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-05T07:44:09.933 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:09 smithi131 ceph-mon[29019]: from='client.? 172.21.15.131:0/3026303119' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-05T07:44:09.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-02-05T07:44:09.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [global] 2024-02-05T07:44:09.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout fsid = 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:44:09.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.131:3300,v1:172.21.15.131:6789] 2024-02-05T07:44:09.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-05T07:44:09.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-05T07:44:09.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-05T07:44:09.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-05T07:44:09.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-02-05T07:44:09.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-05T07:44:09.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-05T07:44:09.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-02-05T07:44:09.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [osd] 2024-02-05T07:44:09.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-05T07:44:09.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-05T07:44:09.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.299+0000 7fe6e7671700 1 Processor -- start 2024-02-05T07:44:09.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.299+0000 7fe6e7671700 1 -- start start 2024-02-05T07:44:09.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.299+0000 7fe6e7671700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6e0105e40 0x7fe6e0106260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:09.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.299+0000 7fe6e7671700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e01067a0 con 0x7fe6e0105e40 2024-02-05T07:44:09.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.299+0000 7fe6e540d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6e0105e40 0x7fe6e0106260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:09.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.299+0000 7fe6e540d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6e0105e40 0x7fe6e0106260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59558/0 (socket says 172.21.15.131:59558) 2024-02-05T07:44:09.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.299+0000 7fe6e540d700 1 -- 172.21.15.131:0/1512697610 learned_addr learned my addr 172.21.15.131:0/1512697610 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:09.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.299+0000 7fe6e540d700 1 -- 172.21.15.131:0/1512697610 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e01068e0 con 0x7fe6e0105e40 2024-02-05T07:44:09.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.300+0000 7fe6e540d700 1 --2- 172.21.15.131:0/1512697610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6e0105e40 0x7fe6e0106260 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0098a0 tx=0x7fe6dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c586c4550f6f9aac server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:09.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.300+0000 7fe6d7fff700 1 -- 172.21.15.131:0/1512697610 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6dc004160 con 0x7fe6e0105e40 2024-02-05T07:44:09.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.300+0000 7fe6d7fff700 1 -- 172.21.15.131:0/1512697610 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6dc0042c0 con 0x7fe6e0105e40 2024-02-05T07:44:09.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.300+0000 7fe6d7fff700 1 -- 172.21.15.131:0/1512697610 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6dc0045a0 con 0x7fe6e0105e40 2024-02-05T07:44:09.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.300+0000 7fe6e7671700 1 -- 172.21.15.131:0/1512697610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6e0105e40 msgr2=0x7fe6e0106260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:09.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.300+0000 7fe6e7671700 1 --2- 172.21.15.131:0/1512697610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6e0105e40 0x7fe6e0106260 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0098a0 tx=0x7fe6dc009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:09.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.300+0000 7fe6e7671700 1 -- 172.21.15.131:0/1512697610 shutdown_connections 2024-02-05T07:44:09.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.300+0000 7fe6e7671700 1 --2- 172.21.15.131:0/1512697610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6e0105e40 0x7fe6e0106260 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:09.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.301+0000 7fe6e7671700 1 -- 172.21.15.131:0/1512697610 >> 172.21.15.131:0/1512697610 conn(0x7fe6e01013a0 msgr2=0x7fe6e0103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:09.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.301+0000 7fe6e7671700 1 -- 172.21.15.131:0/1512697610 shutdown_connections 2024-02-05T07:44:09.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.301+0000 7fe6e7671700 1 -- 172.21.15.131:0/1512697610 wait complete. 2024-02-05T07:44:09.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.301+0000 7fe6e7671700 1 Processor -- start 2024-02-05T07:44:09.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.301+0000 7fe6e7671700 1 -- start start 2024-02-05T07:44:09.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.301+0000 7fe6e7671700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6e0105e40 0x7fe6e0199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:09.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.301+0000 7fe6e7671700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e0199750 con 0x7fe6e0105e40 2024-02-05T07:44:09.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.302+0000 7fe6e540d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6e0105e40 0x7fe6e0199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:09.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.302+0000 7fe6e540d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6e0105e40 0x7fe6e0199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59562/0 (socket says 172.21.15.131:59562) 2024-02-05T07:44:09.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.302+0000 7fe6e540d700 1 -- 172.21.15.131:0/3026303119 learned_addr learned my addr 172.21.15.131:0/3026303119 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:09.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.302+0000 7fe6e540d700 1 -- 172.21.15.131:0/3026303119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6dc009550 con 0x7fe6e0105e40 2024-02-05T07:44:09.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.302+0000 7fe6e540d700 1 --2- 172.21.15.131:0/3026303119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6e0105e40 0x7fe6e0199210 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0106ad0 tx=0x7fe6dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:09.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.302+0000 7fe6d67fc700 1 -- 172.21.15.131:0/3026303119 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6dc011630 con 0x7fe6e0105e40 2024-02-05T07:44:09.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.302+0000 7fe6d67fc700 1 -- 172.21.15.131:0/3026303119 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6dc011790 con 0x7fe6e0105e40 2024-02-05T07:44:09.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.302+0000 7fe6e7671700 1 -- 172.21.15.131:0/3026303119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e0199950 con 0x7fe6e0105e40 2024-02-05T07:44:09.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.302+0000 7fe6d67fc700 1 -- 172.21.15.131:0/3026303119 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6dc011a30 con 0x7fe6e0105e40 2024-02-05T07:44:09.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.302+0000 7fe6e7671700 1 -- 172.21.15.131:0/3026303119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e0199df0 con 0x7fe6e0105e40 2024-02-05T07:44:09.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.302+0000 7fe6d67fc700 1 -- 172.21.15.131:0/3026303119 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe6dc022070 con 0x7fe6e0105e40 2024-02-05T07:44:09.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.302+0000 7fe6d67fc700 1 -- 172.21.15.131:0/3026303119 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe6dc01c450 con 0x7fe6e0105e40 2024-02-05T07:44:09.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.303+0000 7fe6e7671700 1 -- 172.21.15.131:0/3026303119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e0192eb0 con 0x7fe6e0105e40 2024-02-05T07:44:09.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.305+0000 7fe6d67fc700 1 -- 172.21.15.131:0/3026303119 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe6dc011b90 con 0x7fe6e0105e40 2024-02-05T07:44:09.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.375+0000 7fe6e7671700 1 -- 172.21.15.131:0/3026303119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe6e01072d0 con 0x7fe6e0105e40 2024-02-05T07:44:09.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.378+0000 7fe6d67fc700 1 -- 172.21.15.131:0/3026303119 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+340 (secure 0 0 0) 0x7fe6dc011de0 con 0x7fe6e0105e40 2024-02-05T07:44:09.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.379+0000 7fe6d67fc700 1 -- 172.21.15.131:0/3026303119 <== mon.0 v2:172.21.15.131:3300/0 8 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fe6dc01c660 con 0x7fe6e0105e40 2024-02-05T07:44:09.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.379+0000 7fe6e7671700 1 -- 172.21.15.131:0/3026303119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6e0105e40 msgr2=0x7fe6e0199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:09.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.379+0000 7fe6e7671700 1 --2- 172.21.15.131:0/3026303119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6e0105e40 0x7fe6e0199210 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0106ad0 tx=0x7fe6dc004750 comp rx=0 tx=0).stop 2024-02-05T07:44:09.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.380+0000 7fe6e7671700 1 -- 172.21.15.131:0/3026303119 shutdown_connections 2024-02-05T07:44:09.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.380+0000 7fe6e7671700 1 --2- 172.21.15.131:0/3026303119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6e0105e40 0x7fe6e0199210 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:09.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.380+0000 7fe6e7671700 1 -- 172.21.15.131:0/3026303119 >> 172.21.15.131:0/3026303119 conn(0x7fe6e01013a0 msgr2=0x7fe6e018fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:09.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.380+0000 7fe6e7671700 1 -- 172.21.15.131:0/3026303119 shutdown_connections 2024-02-05T07:44:09.946 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:09.380+0000 7fe6e7671700 1 -- 172.21.15.131:0/3026303119 wait complete. 2024-02-05T07:44:09.947 INFO:teuthology.orchestra.run.smithi131.stdout:Generating new minimal ceph.conf... 2024-02-05T07:44:10.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:10 smithi131 ceph-mon[29019]: from='client.? 172.21.15.131:0/180369216' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:44:11.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.402+0000 7f84f26df700 1 Processor -- start 2024-02-05T07:44:11.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.402+0000 7f84f26df700 1 -- start start 2024-02-05T07:44:11.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.402+0000 7f84f26df700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec105e60 0x7f84ec106280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:11.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.402+0000 7f84f26df700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec1067c0 con 0x7f84ec105e60 2024-02-05T07:44:11.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.403+0000 7f84ebfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec105e60 0x7f84ec106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:11.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.403+0000 7f84ebfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec105e60 0x7f84ec106280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59572/0 (socket says 172.21.15.131:59572) 2024-02-05T07:44:11.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.403+0000 7f84ebfff700 1 -- 172.21.15.131:0/488404725 learned_addr learned my addr 172.21.15.131:0/488404725 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:11.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.403+0000 7f84ebfff700 1 -- 172.21.15.131:0/488404725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ec106900 con 0x7f84ec105e60 2024-02-05T07:44:11.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.403+0000 7f84ebfff700 1 --2- 172.21.15.131:0/488404725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec105e60 0x7f84ec106280 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0098a0 tx=0x7f84dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a274dde2c5c734 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:11.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.403+0000 7f84eaffd700 1 -- 172.21.15.131:0/488404725 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84dc004160 con 0x7f84ec105e60 2024-02-05T07:44:11.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.403+0000 7f84eaffd700 1 -- 172.21.15.131:0/488404725 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f84dc0042c0 con 0x7f84ec105e60 2024-02-05T07:44:11.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.403+0000 7f84eaffd700 1 -- 172.21.15.131:0/488404725 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84dc0119b0 con 0x7f84ec105e60 2024-02-05T07:44:11.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.404+0000 7f84f26df700 1 -- 172.21.15.131:0/488404725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec105e60 msgr2=0x7f84ec106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:11.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.404+0000 7f84f26df700 1 --2- 172.21.15.131:0/488404725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec105e60 0x7f84ec106280 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0098a0 tx=0x7f84dc009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:11.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.404+0000 7f84f26df700 1 -- 172.21.15.131:0/488404725 shutdown_connections 2024-02-05T07:44:11.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.404+0000 7f84f26df700 1 --2- 172.21.15.131:0/488404725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec105e60 0x7f84ec106280 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:11.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.404+0000 7f84f26df700 1 -- 172.21.15.131:0/488404725 >> 172.21.15.131:0/488404725 conn(0x7f84ec1014d0 msgr2=0x7f84ec1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:11.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.404+0000 7f84f26df700 1 -- 172.21.15.131:0/488404725 shutdown_connections 2024-02-05T07:44:11.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.404+0000 7f84f26df700 1 -- 172.21.15.131:0/488404725 wait complete. 2024-02-05T07:44:11.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.404+0000 7f84f26df700 1 Processor -- start 2024-02-05T07:44:11.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.404+0000 7f84f26df700 1 -- start start 2024-02-05T07:44:11.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.405+0000 7f84f26df700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec105e60 0x7f84ec199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:11.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.405+0000 7f84f26df700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec19a050 con 0x7f84ec105e60 2024-02-05T07:44:11.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.405+0000 7f84ebfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec105e60 0x7f84ec199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:11.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.405+0000 7f84ebfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec105e60 0x7f84ec199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59574/0 (socket says 172.21.15.131:59574) 2024-02-05T07:44:11.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.405+0000 7f84ebfff700 1 -- 172.21.15.131:0/180369216 learned_addr learned my addr 172.21.15.131:0/180369216 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:11.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.405+0000 7f84ebfff700 1 -- 172.21.15.131:0/180369216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc009550 con 0x7f84ec105e60 2024-02-05T07:44:11.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.405+0000 7f84ebfff700 1 --2- 172.21.15.131:0/180369216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec105e60 0x7f84ec199b10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f84dc006af0 tx=0x7f84dc011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:11.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.405+0000 7f84e97fa700 1 -- 172.21.15.131:0/180369216 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84dc0039e0 con 0x7f84ec105e60 2024-02-05T07:44:11.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.405+0000 7f84e97fa700 1 -- 172.21.15.131:0/180369216 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f84dc003b40 con 0x7f84ec105e60 2024-02-05T07:44:11.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.405+0000 7f84f26df700 1 -- 172.21.15.131:0/180369216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84ec19a250 con 0x7f84ec105e60 2024-02-05T07:44:11.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.405+0000 7f84e97fa700 1 -- 172.21.15.131:0/180369216 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84dc019700 con 0x7f84ec105e60 2024-02-05T07:44:11.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.405+0000 7f84f26df700 1 -- 172.21.15.131:0/180369216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ec19a6f0 con 0x7f84ec105e60 2024-02-05T07:44:11.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.405+0000 7f84e97fa700 1 -- 172.21.15.131:0/180369216 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f84dc019ce0 con 0x7f84ec105e60 2024-02-05T07:44:11.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.405+0000 7f84e97fa700 1 -- 172.21.15.131:0/180369216 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f84dc02c6a0 con 0x7f84ec105e60 2024-02-05T07:44:11.056 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.406+0000 7f84f26df700 1 -- 172.21.15.131:0/180369216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84ec063350 con 0x7f84ec105e60 2024-02-05T07:44:11.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.408+0000 7f84e97fa700 1 -- 172.21.15.131:0/180369216 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f84dc042dd0 con 0x7f84ec105e60 2024-02-05T07:44:11.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.477+0000 7f84f26df700 1 -- 172.21.15.131:0/180369216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f84ec19d000 con 0x7f84ec105e60 2024-02-05T07:44:11.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.477+0000 7f84e97fa700 1 -- 172.21.15.131:0/180369216 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f84dc031020 con 0x7f84ec105e60 2024-02-05T07:44:11.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.478+0000 7f84f26df700 1 -- 172.21.15.131:0/180369216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec105e60 msgr2=0x7f84ec199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:11.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.478+0000 7f84f26df700 1 --2- 172.21.15.131:0/180369216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec105e60 0x7f84ec199b10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f84dc006af0 tx=0x7f84dc011a50 comp rx=0 tx=0).stop 2024-02-05T07:44:11.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.478+0000 7f84f26df700 1 -- 172.21.15.131:0/180369216 shutdown_connections 2024-02-05T07:44:11.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.478+0000 7f84f26df700 1 --2- 172.21.15.131:0/180369216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec105e60 0x7f84ec199b10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:11.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.478+0000 7f84f26df700 1 -- 172.21.15.131:0/180369216 >> 172.21.15.131:0/180369216 conn(0x7f84ec1014d0 msgr2=0x7f84ec1020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:11.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.479+0000 7f84f26df700 1 -- 172.21.15.131:0/180369216 shutdown_connections 2024-02-05T07:44:11.057 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:10.479+0000 7f84f26df700 1 -- 172.21.15.131:0/180369216 wait complete. 2024-02-05T07:44:11.057 INFO:teuthology.orchestra.run.smithi131.stdout:Restarting the monitor... 2024-02-05T07:44:11.304 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:11 smithi131 systemd[1]: Stopping Ceph mon.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T07:44:11.717 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:11 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a[28996]: 2024-02-05T07:44:11.302+0000 7f3a4cba9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T07:44:11.717 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:11 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a[28996]: 2024-02-05T07:44:11.302+0000 7f3a4cba9700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-05T07:44:11.717 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:11 smithi131 podman[29350]: 2024-02-05 07:44:11.476527752 +0000 UTC m=+0.258780896 container died 72f3a427dabef48bbd524370cc33a8bd863be84f1729e8057123e447515b7989 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T07:44:11.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:11 smithi131 podman[29350]: 2024-02-05 07:44:11.81615515 +0000 UTC m=+0.598408297 container cleanup 72f3a427dabef48bbd524370cc33a8bd863be84f1729e8057123e447515b7989 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS) 2024-02-05T07:44:11.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:11 smithi131 bash[29350]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a 2024-02-05T07:44:12.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:11 smithi131 podman[29369]: 2024-02-05 07:44:11.991894374 +0000 UTC m=+0.514211797 container remove 72f3a427dabef48bbd524370cc33a8bd863be84f1729e8057123e447515b7989 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=) 2024-02-05T07:44:12.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-05T07:44:12.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 systemd[1]: Stopped Ceph mon.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T07:44:12.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 systemd[1]: Starting Ceph mon.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T07:44:12.695 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 podman[29469]: 2024-02-05 07:44:12.426135862 +0000 UTC m=+0.107762073 container create 4dc9ef72f635b54c7a848860dca95cb156f97b215dc909a1d35e429e6a9afbcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux ) 2024-02-05T07:44:12.695 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 podman[29469]: 2024-02-05 07:44:12.33591598 +0000 UTC m=+0.017542199 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:44:12.695 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 podman[29469]: 2024-02-05 07:44:12.611674733 +0000 UTC m=+0.293300944 container init 4dc9ef72f635b54c7a848860dca95cb156f97b215dc909a1d35e429e6a9afbcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_CLEAN=True, io.buildah.version=1.33.2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T07:44:12.695 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 podman[29469]: 2024-02-05 07:44:12.616571074 +0000 UTC m=+0.298197288 container start 4dc9ef72f635b54c7a848860dca95cb156f97b215dc909a1d35e429e6a9afbcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-05T07:44:12.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: set uid:gid to 167:167 (ceph:ceph) 2024-02-05T07:44:12.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev), process ceph-mon, pid 7 2024-02-05T07:44:12.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: pidfile_write: ignore empty --pid-file 2024-02-05T07:44:12.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: load: jerasure load: lrc 2024-02-05T07:44:12.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: RocksDB version: 7.9.2 2024-02-05T07:44:12.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Git sha 0 2024-02-05T07:44:12.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Compile date 2024-02-04 23:11:06 2024-02-05T07:44:12.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: DB SUMMARY 2024-02-05T07:44:12.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: DB Session ID: 2Q09D9GVIK0P6IUEIQBL 2024-02-05T07:44:12.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: CURRENT file: CURRENT 2024-02-05T07:44:12.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: IDENTITY file: IDENTITY 2024-02-05T07:44:12.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-05T07:44:12.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-05T07:44:12.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 87727 ; 2024-02-05T07:44:12.696 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.error_if_exists: 0 2024-02-05T07:44:12.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.create_if_missing: 0 2024-02-05T07:44:12.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.paranoid_checks: 1 2024-02-05T07:44:12.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-05T07:44:12.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-05T07:44:12.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-05T07:44:12.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.env: 0x55fd4d1abae0 2024-02-05T07:44:12.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.fs: PosixFileSystem 2024-02-05T07:44:12.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.info_log: 0x55fd4e4ab8c0 2024-02-05T07:44:12.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_file_opening_threads: 16 2024-02-05T07:44:12.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.statistics: (nil) 2024-02-05T07:44:12.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.use_fsync: 0 2024-02-05T07:44:12.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_log_file_size: 0 2024-02-05T07:44:12.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-05T07:44:12.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-05T07:44:12.697 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.keep_log_file_num: 1000 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.recycle_log_file_num: 0 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.allow_fallocate: 1 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.allow_mmap_reads: 0 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.allow_mmap_writes: 0 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.use_direct_reads: 0 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.create_missing_column_families: 0 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.db_log_dir: 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.wal_dir: 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.advise_random_on_open: 1 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.db_write_buffer_size: 0 2024-02-05T07:44:12.698 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.write_buffer_manager: 0x55fd4e47f400 2024-02-05T07:44:12.699 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-05T07:44:12.699 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-05T07:44:12.699 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-05T07:44:12.699 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.rate_limiter: (nil) 2024-02-05T07:44:12.699 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-05T07:44:12.700 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.wal_recovery_mode: 2 2024-02-05T07:44:12.700 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.enable_thread_tracking: 0 2024-02-05T07:44:12.700 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.enable_pipelined_write: 0 2024-02-05T07:44:12.700 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.unordered_write: 0 2024-02-05T07:44:12.700 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-05T07:44:12.700 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-05T07:44:12.700 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-05T07:44:12.700 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-05T07:44:12.700 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.row_cache: None 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.wal_filter: None 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.allow_ingest_behind: 0 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.two_write_queues: 0 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.manual_wal_flush: 0 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.wal_compression: 0 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.atomic_flush: 0 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.log_readahead_size: 0 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.best_efforts_recovery: 0 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.allow_data_in_errors: 0 2024-02-05T07:44:12.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.db_host_id: __hostname__ 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_background_jobs: 2 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_background_compactions: -1 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_subcompactions: 1 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_total_wal_size: 0 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_open_files: -1 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.bytes_per_sync: 0 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-05T07:44:12.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compaction_readahead_size: 0 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_background_flushes: -1 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Compression algorithms supported: 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: kZSTD supported: 0 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: kXpressCompression supported: 0 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: kLZ4HCCompression supported: 1 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: kZlibCompression supported: 1 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: kSnappyCompression supported: 1 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: kLZ4Compression supported: 1 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: kBZip2Compression supported: 0 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.merge_operator: 2024-02-05T07:44:12.703 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compaction_filter: None 2024-02-05T07:44:12.704 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compaction_filter_factory: None 2024-02-05T07:44:12.704 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.sst_partitioner_factory: None 2024-02-05T07:44:12.704 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-05T07:44:12.704 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-05T07:44:12.704 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fd4e4ab9c0) 2024-02-05T07:44:12.704 INFO:journalctl@ceph.mon.a.smithi131.stdout: cache_index_and_filter_blocks: 1 2024-02-05T07:44:12.704 INFO:journalctl@ceph.mon.a.smithi131.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-05T07:44:12.704 INFO:journalctl@ceph.mon.a.smithi131.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-05T07:44:12.704 INFO:journalctl@ceph.mon.a.smithi131.stdout: pin_top_level_index_and_filter: 1 2024-02-05T07:44:12.704 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_type: 0 2024-02-05T07:44:12.704 INFO:journalctl@ceph.mon.a.smithi131.stdout: data_block_index_type: 0 2024-02-05T07:44:12.704 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_shortening: 1 2024-02-05T07:44:12.704 INFO:journalctl@ceph.mon.a.smithi131.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-05T07:44:12.704 INFO:journalctl@ceph.mon.a.smithi131.stdout: checksum: 4 2024-02-05T07:44:12.705 INFO:journalctl@ceph.mon.a.smithi131.stdout: no_block_cache: 0 2024-02-05T07:44:12.705 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache: 0x55fd4e4af090 2024-02-05T07:44:12.705 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_name: BinnedLRUCache 2024-02-05T07:44:12.705 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_options: 2024-02-05T07:44:12.705 INFO:journalctl@ceph.mon.a.smithi131.stdout: capacity : 536870912 2024-02-05T07:44:12.705 INFO:journalctl@ceph.mon.a.smithi131.stdout: num_shard_bits : 4 2024-02-05T07:44:12.705 INFO:journalctl@ceph.mon.a.smithi131.stdout: strict_capacity_limit : 0 2024-02-05T07:44:12.705 INFO:journalctl@ceph.mon.a.smithi131.stdout: high_pri_pool_ratio: 0.000 2024-02-05T07:44:12.705 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_compressed: (nil) 2024-02-05T07:44:12.705 INFO:journalctl@ceph.mon.a.smithi131.stdout: persistent_cache: (nil) 2024-02-05T07:44:12.705 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_size: 4096 2024-02-05T07:44:12.705 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_size_deviation: 10 2024-02-05T07:44:12.705 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_restart_interval: 16 2024-02-05T07:44:12.705 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_block_restart_interval: 1 2024-02-05T07:44:12.705 INFO:journalctl@ceph.mon.a.smithi131.stdout: metadata_block_size: 4096 2024-02-05T07:44:12.705 INFO:journalctl@ceph.mon.a.smithi131.stdout: partition_filters: 0 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout: use_delta_encoding: 1 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout: filter_policy: bloomfilter 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout: whole_key_filtering: 1 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout: verify_compression: 0 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout: read_amp_bytes_per_bit: 0 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout: format_version: 5 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout: enable_index_compression: 1 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_align: 0 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout: max_auto_readahead_size: 262144 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout: prepopulate_block_cache: 0 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout: initial_auto_readahead_size: 8192 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout: num_file_reads_for_auto_readahead: 2 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.write_buffer_size: 33554432 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_write_buffer_number: 2 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compression: NoCompression 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.bottommost_compression: Disabled 2024-02-05T07:44:12.706 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.prefix_extractor: nullptr 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.num_levels: 7 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compression_opts.level: 32767 2024-02-05T07:44:12.707 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compression_opts.strategy: 0 2024-02-05T07:44:12.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-05T07:44:12.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-05T07:44:12.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-05T07:44:12.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-05T07:44:12.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compression_opts.enabled: false 2024-02-05T07:44:12.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-05T07:44:12.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-05T07:44:12.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-05T07:44:12.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-05T07:44:12.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.target_file_size_base: 67108864 2024-02-05T07:44:12.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-05T07:44:12.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-05T07:44:12.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-05T07:44:12.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-05T07:44:12.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-05T07:44:12.708 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.arena_block_size: 1048576 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.disable_auto_compactions: 0 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-05T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.table_properties_collectors: 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.inplace_update_support: 0 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.bloom_locality: 0 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.max_successive_merges: 0 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.paranoid_file_checks: 0 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.force_consistency_checks: 1 2024-02-05T07:44:12.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.report_bg_io_stats: 0 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.ttl: 2592000 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.enable_blob_files: false 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.min_blob_size: 0 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.blob_file_size: 268435456 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.blob_file_starting_level: 0 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-05T07:44:12.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c8c99040-df5a-4672-b283-613c164c913b 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707119052646064, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707119052646901, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 83975, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 266, "table_properties": {"data_size": 82154, "index_size": 231, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10814, "raw_average_key_size": 48, "raw_value_size": 76131, "raw_average_value_size": 338, "num_data_blocks": 10, "num_entries": 225, "num_filter_entries": 225, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707119052, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c8c99040-df5a-4672-b283-613c164c913b", "db_session_id": "2Q09D9GVIK0P6IUEIQBL", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707119052646982, "job": 1, "event": "recovery_finished"} 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55fd4e5a0000 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: DB pointer 0x55fd4e58a000 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** DB Stats ** 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-05T07:44:12.712 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** Compaction Stats [default] ** 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: L0 2/0 83.84 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 152.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: Sum 2/0 83.84 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 152.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 152.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** Compaction Stats [default] ** 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 152.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-02-05T07:44:12.713 INFO:journalctl@ceph.mon.a.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative compaction: 0.00 GB write, 28.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval compaction: 0.00 GB write, 28.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout: Block cache BinnedLRUCache@0x55fd4e4af090#7 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: starting mon.a rank 0 at public addrs [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] at bind addrs [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: mon.a@-1(???) e1 preinit fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: mon.a@-1(???).mds e1 new map 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: mon.a@-1(???).mds e1 print_map 2024-02-05T07:44:12.714 INFO:journalctl@ceph.mon.a.smithi131.stdout: e1 2024-02-05T07:44:12.715 INFO:journalctl@ceph.mon.a.smithi131.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-05T07:44:12.715 INFO:journalctl@ceph.mon.a.smithi131.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-05T07:44:12.715 INFO:journalctl@ceph.mon.a.smithi131.stdout: legacy client fscid: -1 2024-02-05T07:44:12.715 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-02-05T07:44:12.715 INFO:journalctl@ceph.mon.a.smithi131.stdout: No filesystems configured 2024-02-05T07:44:12.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-05T07:44:12.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T07:44:12.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T07:44:12.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T07:44:12.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-05T07:44:12.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 bash[29469]: 4dc9ef72f635b54c7a848860dca95cb156f97b215dc909a1d35e429e6a9afbcf 2024-02-05T07:44:12.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 systemd[1]: Started Ceph mon.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T07:44:12.715 INFO:teuthology.orchestra.run.smithi131.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-05T07:44:12.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-05T07:44:12.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: monmap e1: 1 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T07:44:12.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: fsmap 2024-02-05T07:44:12.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: osdmap e1: 0 total, 0 up, 0 in 2024-02-05T07:44:12.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:12 smithi131 ceph-mon[29507]: mgrmap e1: no daemons active 2024-02-05T07:44:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.210+0000 7efe3b4fa700 1 Processor -- start 2024-02-05T07:44:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.210+0000 7efe3b4fa700 1 -- start start 2024-02-05T07:44:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.210+0000 7efe3b4fa700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe34105e60 0x7efe34106280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.210+0000 7efe3b4fa700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe341067c0 con 0x7efe34105e60 2024-02-05T07:44:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.211+0000 7efe39296700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe34105e60 0x7efe34106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.211+0000 7efe39296700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe34105e60 0x7efe34106280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59586/0 (socket says 172.21.15.131:59586) 2024-02-05T07:44:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.211+0000 7efe39296700 1 -- 172.21.15.131:0/1589832619 learned_addr learned my addr 172.21.15.131:0/1589832619 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.211+0000 7efe39296700 1 -- 172.21.15.131:0/1589832619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe34106900 con 0x7efe34105e60 2024-02-05T07:44:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.211+0000 7efe39296700 1 --2- 172.21.15.131:0/1589832619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe34105e60 0x7efe34106280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efe300098a0 tx=0x7efe30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4c817972616b09 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.211+0000 7efe2bfff700 1 -- 172.21.15.131:0/1589832619 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe30004160 con 0x7efe34105e60 2024-02-05T07:44:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.211+0000 7efe2bfff700 1 -- 172.21.15.131:0/1589832619 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7efe300042c0 con 0x7efe34105e60 2024-02-05T07:44:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.211+0000 7efe2bfff700 1 -- 172.21.15.131:0/1589832619 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe300119b0 con 0x7efe34105e60 2024-02-05T07:44:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.212+0000 7efe3b4fa700 1 -- 172.21.15.131:0/1589832619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe34105e60 msgr2=0x7efe34106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.212+0000 7efe3b4fa700 1 --2- 172.21.15.131:0/1589832619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe34105e60 0x7efe34106280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efe300098a0 tx=0x7efe30009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.212+0000 7efe3b4fa700 1 -- 172.21.15.131:0/1589832619 shutdown_connections 2024-02-05T07:44:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.212+0000 7efe3b4fa700 1 --2- 172.21.15.131:0/1589832619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe34105e60 0x7efe34106280 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.212+0000 7efe3b4fa700 1 -- 172.21.15.131:0/1589832619 >> 172.21.15.131:0/1589832619 conn(0x7efe341014d0 msgr2=0x7efe341038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.212+0000 7efe3b4fa700 1 -- 172.21.15.131:0/1589832619 shutdown_connections 2024-02-05T07:44:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.212+0000 7efe3b4fa700 1 -- 172.21.15.131:0/1589832619 wait complete. 2024-02-05T07:44:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.213+0000 7efe3b4fa700 1 Processor -- start 2024-02-05T07:44:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.213+0000 7efe3b4fa700 1 -- start start 2024-02-05T07:44:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.213+0000 7efe3b4fa700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe34105e60 0x7efe34199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.213+0000 7efe3b4fa700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3419a050 con 0x7efe34105e60 2024-02-05T07:44:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.213+0000 7efe39296700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe34105e60 0x7efe34199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.213+0000 7efe39296700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe34105e60 0x7efe34199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59594/0 (socket says 172.21.15.131:59594) 2024-02-05T07:44:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.213+0000 7efe39296700 1 -- 172.21.15.131:0/1662328581 learned_addr learned my addr 172.21.15.131:0/1662328581 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.213+0000 7efe39296700 1 -- 172.21.15.131:0/1662328581 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe30009550 con 0x7efe34105e60 2024-02-05T07:44:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.213+0000 7efe39296700 1 --2- 172.21.15.131:0/1662328581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe34105e60 0x7efe34199b10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efe30006af0 tx=0x7efe30011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.213+0000 7efe2a7fc700 1 -- 172.21.15.131:0/1662328581 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe300039e0 con 0x7efe34105e60 2024-02-05T07:44:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.213+0000 7efe2a7fc700 1 -- 172.21.15.131:0/1662328581 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7efe30003b40 con 0x7efe34105e60 2024-02-05T07:44:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.213+0000 7efe3b4fa700 1 -- 172.21.15.131:0/1662328581 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3419a250 con 0x7efe34105e60 2024-02-05T07:44:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.213+0000 7efe2a7fc700 1 -- 172.21.15.131:0/1662328581 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe30019700 con 0x7efe34105e60 2024-02-05T07:44:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.214+0000 7efe3b4fa700 1 -- 172.21.15.131:0/1662328581 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3419a6f0 con 0x7efe34105e60 2024-02-05T07:44:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.214+0000 7efe2a7fc700 1 -- 172.21.15.131:0/1662328581 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7efe30019cc0 con 0x7efe34105e60 2024-02-05T07:44:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.214+0000 7efe2a7fc700 1 -- 172.21.15.131:0/1662328581 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efe3002c660 con 0x7efe34105e60 2024-02-05T07:44:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.214+0000 7efe3b4fa700 1 -- 172.21.15.131:0/1662328581 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe34063350 con 0x7efe34105e60 2024-02-05T07:44:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.217+0000 7efe2a7fc700 1 -- 172.21.15.131:0/1662328581 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7efe30042dd0 con 0x7efe34105e60 2024-02-05T07:44:13.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.283+0000 7efe3b4fa700 1 -- 172.21.15.131:0/1662328581 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7efe3419d070 con 0x7efe34105e60 2024-02-05T07:44:13.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.285+0000 7efe2a7fc700 1 -- 172.21.15.131:0/1662328581 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7efe30031020 con 0x7efe34105e60 2024-02-05T07:44:13.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.286+0000 7efe3b4fa700 1 -- 172.21.15.131:0/1662328581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe34105e60 msgr2=0x7efe34199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:13.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.286+0000 7efe3b4fa700 1 --2- 172.21.15.131:0/1662328581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe34105e60 0x7efe34199b10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efe30006af0 tx=0x7efe30011a50 comp rx=0 tx=0).stop 2024-02-05T07:44:13.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.286+0000 7efe3b4fa700 1 -- 172.21.15.131:0/1662328581 shutdown_connections 2024-02-05T07:44:13.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.286+0000 7efe3b4fa700 1 --2- 172.21.15.131:0/1662328581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe34105e60 0x7efe34199b10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:13.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.286+0000 7efe3b4fa700 1 -- 172.21.15.131:0/1662328581 >> 172.21.15.131:0/1662328581 conn(0x7efe341014d0 msgr2=0x7efe341020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:13.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.286+0000 7efe3b4fa700 1 -- 172.21.15.131:0/1662328581 shutdown_connections 2024-02-05T07:44:13.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:13.286+0000 7efe3b4fa700 1 -- 172.21.15.131:0/1662328581 wait complete. 2024-02-05T07:44:13.945 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-05T07:44:13.967 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-05T07:44:13.967 INFO:teuthology.orchestra.run.smithi131.stdout:Creating mgr... 2024-02-05T07:44:13.968 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-05T07:44:13.968 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-05T07:44:14.224 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mgr.y 2024-02-05T07:44:14.224 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mgr.y.service: Unit ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mgr.y.service not loaded. 2024-02-05T07:44:14.375 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715.target.wants/ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@.service. 2024-02-05T07:44:14.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1662328581' entity='client.admin' 2024-02-05T07:44:14.518 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:14 smithi131 systemd[1]: Starting Ceph mgr.y for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T07:44:14.939 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:14 smithi131 podman[29752]: 2024-02-05 07:44:14.512541919 +0000 UTC m=+0.017333829 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:44:14.939 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:14 smithi131 podman[29752]: 2024-02-05 07:44:14.619384418 +0000 UTC m=+0.124176364 container create afc5c66e51ee13b90ceb6967acb93cbd2902010f2fb0fa9d0e5b4c8ef7916506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , ceph=True) 2024-02-05T07:44:14.939 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:14 smithi131 podman[29752]: 2024-02-05 07:44:14.846568098 +0000 UTC m=+0.351360002 container init afc5c66e51ee13b90ceb6967acb93cbd2902010f2fb0fa9d0e5b4c8ef7916506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-05T07:44:14.939 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:14 smithi131 podman[29752]: 2024-02-05 07:44:14.852026459 +0000 UTC m=+0.356818364 container start afc5c66e51ee13b90ceb6967acb93cbd2902010f2fb0fa9d0e5b4c8ef7916506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-05T07:44:14.939 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:14 smithi131 bash[29752]: afc5c66e51ee13b90ceb6967acb93cbd2902010f2fb0fa9d0e5b4c8ef7916506 2024-02-05T07:44:14.952 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-05T07:44:14.952 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-02-05T07:44:14.959 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-05T07:44:14.959 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-02-05T07:44:14.959 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-02-05T07:44:14.959 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to enable service . firewalld.service is not available 2024-02-05T07:44:14.968 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-05T07:44:14.968 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-02-05T07:44:14.975 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-05T07:44:14.975 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-02-05T07:44:14.975 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-02-05T07:44:14.975 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-05T07:44:14.975 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr to start... 2024-02-05T07:44:14.975 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr... 2024-02-05T07:44:15.218 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:14 smithi131 systemd[1]: Started Ceph mgr.y for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T07:44:15.468 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:15 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:15.405+0000 7f4f39ff5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T07:44:15.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2075228629' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-05T07:44:15.768 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:15 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:15.502+0000 7f4f39ff5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T07:44:16.148 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-02-05T07:44:16.151 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-02-05T07:44:16.151 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "4018fbe8-c3fa-11ee-95b6-87774f69a715", 2024-02-05T07:44:16.151 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-02-05T07:44:16.151 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-05T07:44:16.151 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-05T07:44:16.151 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-05T07:44:16.151 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:16.151 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-05T07:44:16.151 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-05T07:44:16.152 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-02-05T07:44:16.152 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:44:16.152 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-05T07:44:16.152 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-02-05T07:44:16.152 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:44:16.152 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-05T07:44:16.152 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-05T07:44:16.152 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:44:16.152 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-05T07:44:16.152 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-05T07:44:16.153 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:16.153 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-05T07:44:16.153 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:44:16.153 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-05T07:44:16.153 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-05T07:44:16.153 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-05T07:44:16.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-05T07:44:16.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-05T07:44:16.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-05T07:44:16.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:16.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-05T07:44:16.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-05T07:44:16.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-05T07:44:16.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-05T07:44:16.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-05T07:44:16.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-05T07:44:16.161 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-05T07:44:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-05T07:44:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-05T07:44:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-05T07:44:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:44:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-05T07:44:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-05T07:44:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-05T07:44:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-05T07:44:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-05T07:44:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-05T07:44:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-05T07:44:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-05T07:44:16.162 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-02-05T07:44:16.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:44:16.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T07:44:16.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:16.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-05T07:44:16.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:44:16.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-02-05T07:44:07.661401+0000", 2024-02-05T07:44:16.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T07:44:16.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:16.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-05T07:44:16.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-02-05T07:44:16.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.463+0000 7f3e265f6700 1 Processor -- start 2024-02-05T07:44:16.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.463+0000 7f3e265f6700 1 -- start start 2024-02-05T07:44:16.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.463+0000 7f3e265f6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e20106310 0x7f3e20106730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:16.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.463+0000 7f3e265f6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e20106d00 con 0x7f3e20106310 2024-02-05T07:44:16.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.463+0000 7f3e1ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e20106310 0x7f3e20106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:16.163 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.463+0000 7f3e1ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e20106310 0x7f3e20106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59628/0 (socket says 172.21.15.131:59628) 2024-02-05T07:44:16.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.463+0000 7f3e1ffff700 1 -- 172.21.15.131:0/3746965876 learned_addr learned my addr 172.21.15.131:0/3746965876 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:16.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.464+0000 7f3e1ffff700 1 -- 172.21.15.131:0/3746965876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e20106e40 con 0x7f3e20106310 2024-02-05T07:44:16.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.464+0000 7f3e1ffff700 1 --2- 172.21.15.131:0/3746965876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e20106310 0x7f3e20106730 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3e100098a0 tx=0x7f3e10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c12426895435c36 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:16.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.464+0000 7f3e1effd700 1 -- 172.21.15.131:0/3746965876 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e10004160 con 0x7f3e20106310 2024-02-05T07:44:16.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.464+0000 7f3e1effd700 1 -- 172.21.15.131:0/3746965876 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f3e100042c0 con 0x7f3e20106310 2024-02-05T07:44:16.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.464+0000 7f3e1effd700 1 -- 172.21.15.131:0/3746965876 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e100119b0 con 0x7f3e20106310 2024-02-05T07:44:16.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.465+0000 7f3e265f6700 1 -- 172.21.15.131:0/3746965876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e20106310 msgr2=0x7f3e20106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:16.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.465+0000 7f3e265f6700 1 --2- 172.21.15.131:0/3746965876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e20106310 0x7f3e20106730 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3e100098a0 tx=0x7f3e10009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:16.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.465+0000 7f3e265f6700 1 -- 172.21.15.131:0/3746965876 shutdown_connections 2024-02-05T07:44:16.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.465+0000 7f3e265f6700 1 --2- 172.21.15.131:0/3746965876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e20106310 0x7f3e20106730 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:16.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.465+0000 7f3e265f6700 1 -- 172.21.15.131:0/3746965876 >> 172.21.15.131:0/3746965876 conn(0x7f3e200ffa40 msgr2=0x7f3e20101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:16.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.465+0000 7f3e265f6700 1 -- 172.21.15.131:0/3746965876 shutdown_connections 2024-02-05T07:44:16.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.465+0000 7f3e265f6700 1 -- 172.21.15.131:0/3746965876 wait complete. 2024-02-05T07:44:16.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.465+0000 7f3e265f6700 1 Processor -- start 2024-02-05T07:44:16.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.465+0000 7f3e265f6700 1 -- start start 2024-02-05T07:44:16.164 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.466+0000 7f3e265f6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e20106310 0x7f3e2019bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.466+0000 7f3e265f6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2019c1e0 con 0x7f3e20106310 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.466+0000 7f3e1ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e20106310 0x7f3e2019bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.466+0000 7f3e1ffff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e20106310 0x7f3e2019bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59632/0 (socket says 172.21.15.131:59632) 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.466+0000 7f3e1ffff700 1 -- 172.21.15.131:0/2075228629 learned_addr learned my addr 172.21.15.131:0/2075228629 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.466+0000 7f3e1ffff700 1 -- 172.21.15.131:0/2075228629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e10009550 con 0x7f3e20106310 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.466+0000 7f3e1ffff700 1 --2- 172.21.15.131:0/2075228629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e20106310 0x7f3e2019bca0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3e10006af0 tx=0x7f3e10011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.466+0000 7f3e1d7fa700 1 -- 172.21.15.131:0/2075228629 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e100039e0 con 0x7f3e20106310 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.466+0000 7f3e1d7fa700 1 -- 172.21.15.131:0/2075228629 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f3e10003b40 con 0x7f3e20106310 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.466+0000 7f3e265f6700 1 -- 172.21.15.131:0/2075228629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2019c3e0 con 0x7f3e20106310 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.466+0000 7f3e1d7fa700 1 -- 172.21.15.131:0/2075228629 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e10019720 con 0x7f3e20106310 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.467+0000 7f3e265f6700 1 -- 172.21.15.131:0/2075228629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2019c880 con 0x7f3e20106310 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.467+0000 7f3e1d7fa700 1 -- 172.21.15.131:0/2075228629 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3e10019d50 con 0x7f3e20106310 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.467+0000 7f3e1d7fa700 1 -- 172.21.15.131:0/2075228629 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3e1002c6f0 con 0x7f3e20106310 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.467+0000 7f3e265f6700 1 -- 172.21.15.131:0/2075228629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e20195a00 con 0x7f3e20106310 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.470+0000 7f3e1d7fa700 1 -- 172.21.15.131:0/2075228629 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3e10042dd0 con 0x7f3e20106310 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.543+0000 7f3e265f6700 1 -- 172.21.15.131:0/2075228629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3e20104c20 con 0x7f3e20106310 2024-02-05T07:44:16.165 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.543+0000 7f3e1d7fa700 1 -- 172.21.15.131:0/2075228629 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f3e10031020 con 0x7f3e20106310 2024-02-05T07:44:16.166 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.545+0000 7f3e265f6700 1 -- 172.21.15.131:0/2075228629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e20106310 msgr2=0x7f3e2019bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:16.166 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.545+0000 7f3e265f6700 1 --2- 172.21.15.131:0/2075228629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e20106310 0x7f3e2019bca0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3e10006af0 tx=0x7f3e10011a50 comp rx=0 tx=0).stop 2024-02-05T07:44:16.166 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.545+0000 7f3e265f6700 1 -- 172.21.15.131:0/2075228629 shutdown_connections 2024-02-05T07:44:16.166 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.545+0000 7f3e265f6700 1 --2- 172.21.15.131:0/2075228629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e20106310 0x7f3e2019bca0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:16.166 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.545+0000 7f3e265f6700 1 -- 172.21.15.131:0/2075228629 >> 172.21.15.131:0/2075228629 conn(0x7f3e200ffa40 msgr2=0x7f3e20192b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:16.166 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.545+0000 7f3e265f6700 1 -- 172.21.15.131:0/2075228629 shutdown_connections 2024-02-05T07:44:16.166 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:15.545+0000 7f3e265f6700 1 -- 172.21.15.131:0/2075228629 wait complete. 2024-02-05T07:44:16.166 INFO:teuthology.orchestra.run.smithi131.stdout:mgr not available, waiting (1/15)... 2024-02-05T07:44:16.486 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:16 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:16.158+0000 7f4f39ff5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T07:44:16.487 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:16 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:16.250+0000 7f4f39ff5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T07:44:16.487 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:16 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:16.391+0000 7f4f39ff5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T07:44:16.767 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:16 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:16.485+0000 7f4f39ff5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T07:44:16.768 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:16 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:16.568+0000 7f4f39ff5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T07:44:17.122 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:16 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:16.836+0000 7f4f39ff5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T07:44:17.123 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:16 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:16.945+0000 7f4f39ff5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T07:44:17.123 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:17 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:17.029+0000 7f4f39ff5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T07:44:17.123 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:17 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:17.121+0000 7f4f39ff5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T07:44:17.517 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:17 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:17.300+0000 7f4f39ff5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T07:44:18.521 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:18 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:18.268+0000 7f4f39ff5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T07:44:18.521 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:18 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:18.401+0000 7f4f39ff5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T07:44:18.843 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2753536396' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-05T07:44:18.844 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:18 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:18.753+0000 7f4f39ff5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T07:44:18.844 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:18 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:18.842+0000 7f4f39ff5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T07:44:19.225 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:19 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:19.040+0000 7f4f39ff5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:44:19.225 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:19 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:19.224+0000 7f4f39ff5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T07:44:19.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-02-05T07:44:19.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-02-05T07:44:19.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "4018fbe8-c3fa-11ee-95b6-87774f69a715", 2024-02-05T07:44:19.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-02-05T07:44:19.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-05T07:44:19.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-05T07:44:19.281 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-05T07:44:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-05T07:44:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-05T07:44:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-02-05T07:44:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:44:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-05T07:44:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-02-05T07:44:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:44:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-02-05T07:44:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-05T07:44:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:44:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-05T07:44:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-05T07:44:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-05T07:44:19.282 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:44:19.283 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-05T07:44:19.283 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-05T07:44:19.283 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-05T07:44:19.290 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-05T07:44:19.290 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-05T07:44:19.290 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-05T07:44:19.290 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:19.290 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-05T07:44:19.290 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-05T07:44:19.290 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-05T07:44:19.290 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-05T07:44:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-05T07:44:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-05T07:44:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-05T07:44:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-05T07:44:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-05T07:44:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-05T07:44:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:44:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-05T07:44:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-05T07:44:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-05T07:44:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-05T07:44:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-05T07:44:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-05T07:44:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-05T07:44:19.292 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-05T07:44:19.292 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-02-05T07:44:19.292 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:44:19.292 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T07:44:19.292 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:19.292 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-05T07:44:19.292 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:44:19.292 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-02-05T07:44:07.661401+0000", 2024-02-05T07:44:19.292 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T07:44:19.292 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:19.292 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-05T07:44:19.292 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-02-05T07:44:19.292 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.618+0000 7f7274a1a700 1 Processor -- start 2024-02-05T07:44:19.292 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.618+0000 7f7274a1a700 1 -- start start 2024-02-05T07:44:19.292 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.618+0000 7f7274a1a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7270104d00 0x7f7270105120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:19.293 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.618+0000 7f7274a1a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72701056f0 con 0x7f7270104d00 2024-02-05T07:44:19.293 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.618+0000 7f726e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7270104d00 0x7f7270105120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:19.293 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.618+0000 7f726e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7270104d00 0x7f7270105120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50146/0 (socket says 172.21.15.131:50146) 2024-02-05T07:44:19.293 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.618+0000 7f726e59c700 1 -- 172.21.15.131:0/4173684222 learned_addr learned my addr 172.21.15.131:0/4173684222 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:19.293 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.619+0000 7f726e59c700 1 -- 172.21.15.131:0/4173684222 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7270105f00 con 0x7f7270104d00 2024-02-05T07:44:19.293 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.619+0000 7f726e59c700 1 --2- 172.21.15.131:0/4173684222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7270104d00 0x7f7270105120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f72600098a0 tx=0x7f7260009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf7c33e548a388a server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:19.293 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.619+0000 7f726d59a700 1 -- 172.21.15.131:0/4173684222 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7260004160 con 0x7f7270104d00 2024-02-05T07:44:19.293 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.619+0000 7f726d59a700 1 -- 172.21.15.131:0/4173684222 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f72600042c0 con 0x7f7270104d00 2024-02-05T07:44:19.293 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.619+0000 7f726d59a700 1 -- 172.21.15.131:0/4173684222 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72600119b0 con 0x7f7270104d00 2024-02-05T07:44:19.293 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.620+0000 7f7274a1a700 1 -- 172.21.15.131:0/4173684222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7270104d00 msgr2=0x7f7270105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:19.293 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.620+0000 7f7274a1a700 1 --2- 172.21.15.131:0/4173684222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7270104d00 0x7f7270105120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f72600098a0 tx=0x7f7260009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:19.293 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.620+0000 7f7274a1a700 1 -- 172.21.15.131:0/4173684222 shutdown_connections 2024-02-05T07:44:19.293 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.620+0000 7f7274a1a700 1 --2- 172.21.15.131:0/4173684222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7270104d00 0x7f7270105120 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:19.293 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.620+0000 7f7274a1a700 1 -- 172.21.15.131:0/4173684222 >> 172.21.15.131:0/4173684222 conn(0x7f72701002c0 msgr2=0x7f72701026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:19.293 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.620+0000 7f7274a1a700 1 -- 172.21.15.131:0/4173684222 shutdown_connections 2024-02-05T07:44:19.294 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.620+0000 7f7274a1a700 1 -- 172.21.15.131:0/4173684222 wait complete. 2024-02-05T07:44:19.294 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.621+0000 7f7274a1a700 1 Processor -- start 2024-02-05T07:44:19.294 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.621+0000 7f7274a1a700 1 -- start start 2024-02-05T07:44:19.294 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.621+0000 7f7274a1a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7270104d00 0x7f7270199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:19.294 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.621+0000 7f7274a1a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727019a020 con 0x7f7270104d00 2024-02-05T07:44:19.294 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.621+0000 7f726e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7270104d00 0x7f7270199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:19.294 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.621+0000 7f726e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7270104d00 0x7f7270199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50162/0 (socket says 172.21.15.131:50162) 2024-02-05T07:44:19.294 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.621+0000 7f726e59c700 1 -- 172.21.15.131:0/2753536396 learned_addr learned my addr 172.21.15.131:0/2753536396 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:19.294 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.621+0000 7f726e59c700 1 -- 172.21.15.131:0/2753536396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7260009550 con 0x7f7270104d00 2024-02-05T07:44:19.294 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.621+0000 7f726e59c700 1 --2- 172.21.15.131:0/2753536396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7270104d00 0x7f7270199ae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f72600046c0 tx=0x7f7260011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:19.294 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.621+0000 7f725f7fe700 1 -- 172.21.15.131:0/2753536396 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72600039c0 con 0x7f7270104d00 2024-02-05T07:44:19.294 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.622+0000 7f725f7fe700 1 -- 172.21.15.131:0/2753536396 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f7260003b20 con 0x7f7270104d00 2024-02-05T07:44:19.294 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.622+0000 7f7274a1a700 1 -- 172.21.15.131:0/2753536396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727019a220 con 0x7f7270104d00 2024-02-05T07:44:19.294 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.622+0000 7f725f7fe700 1 -- 172.21.15.131:0/2753536396 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f726001a780 con 0x7f7270104d00 2024-02-05T07:44:19.294 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.622+0000 7f7274a1a700 1 -- 172.21.15.131:0/2753536396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727019a6c0 con 0x7f7270104d00 2024-02-05T07:44:19.295 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.622+0000 7f725f7fe700 1 -- 172.21.15.131:0/2753536396 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f726001a9f0 con 0x7f7270104d00 2024-02-05T07:44:19.295 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.622+0000 7f725f7fe700 1 -- 172.21.15.131:0/2753536396 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f726002de40 con 0x7f7270104d00 2024-02-05T07:44:19.295 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.623+0000 7f7274a1a700 1 -- 172.21.15.131:0/2753536396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7270193bd0 con 0x7f7270104d00 2024-02-05T07:44:19.295 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.625+0000 7f725f7fe700 1 -- 172.21.15.131:0/2753536396 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7260024b50 con 0x7f7270104d00 2024-02-05T07:44:19.295 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.697+0000 7f7274a1a700 1 -- 172.21.15.131:0/2753536396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f72701935b0 con 0x7f7270104d00 2024-02-05T07:44:19.295 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.697+0000 7f725f7fe700 1 -- 172.21.15.131:0/2753536396 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f7260024cf0 con 0x7f7270104d00 2024-02-05T07:44:19.295 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.698+0000 7f7274a1a700 1 -- 172.21.15.131:0/2753536396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7270104d00 msgr2=0x7f7270199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:19.295 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.698+0000 7f7274a1a700 1 --2- 172.21.15.131:0/2753536396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7270104d00 0x7f7270199ae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f72600046c0 tx=0x7f7260011a50 comp rx=0 tx=0).stop 2024-02-05T07:44:19.295 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.698+0000 7f7274a1a700 1 -- 172.21.15.131:0/2753536396 shutdown_connections 2024-02-05T07:44:19.295 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.698+0000 7f7274a1a700 1 --2- 172.21.15.131:0/2753536396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7270104d00 0x7f7270199ae0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:19.295 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.698+0000 7f7274a1a700 1 -- 172.21.15.131:0/2753536396 >> 172.21.15.131:0/2753536396 conn(0x7f72701002c0 msgr2=0x7f72701011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:19.295 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.699+0000 7f7274a1a700 1 -- 172.21.15.131:0/2753536396 shutdown_connections 2024-02-05T07:44:19.295 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:18.699+0000 7f7274a1a700 1 -- 172.21.15.131:0/2753536396 wait complete. 2024-02-05T07:44:19.295 INFO:teuthology.orchestra.run.smithi131.stdout:mgr not available, waiting (2/15)... 2024-02-05T07:44:19.767 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:19 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:19.505+0000 7f4f39ff5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:44:20.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:19 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:19.769+0000 7f4f39ff5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T07:44:20.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:20 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:20.000+0000 7f4f39ff5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T07:44:21.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:20 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:20.802+0000 7f4f39ff5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T07:44:21.668 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:21 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:21.376+0000 7f4f39ff5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T07:44:21.668 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:21 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:21.474+0000 7f4f39ff5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T07:44:22.017 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:21 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:21.694+0000 7f4f39ff5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T07:44:22.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:21 smithi131 ceph-mon[29507]: Activating manager daemon y 2024-02-05T07:44:22.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:21 smithi131 ceph-mon[29507]: mgrmap e2: y(active, starting, since 0.00294187s) 2024-02-05T07:44:22.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:21 smithi131 ceph-mon[29507]: from='mgr.14100 172.21.15.131:0/4046130138' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T07:44:22.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:21 smithi131 ceph-mon[29507]: from='mgr.14100 172.21.15.131:0/4046130138' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T07:44:22.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:21 smithi131 ceph-mon[29507]: from='mgr.14100 172.21.15.131:0/4046130138' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T07:44:22.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:21 smithi131 ceph-mon[29507]: from='mgr.14100 172.21.15.131:0/4046130138' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:44:22.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:21 smithi131 ceph-mon[29507]: from='mgr.14100 172.21.15.131:0/4046130138' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T07:44:22.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:21 smithi131 ceph-mon[29507]: Manager daemon y is now available 2024-02-05T07:44:22.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:21 smithi131 ceph-mon[29507]: from='mgr.14100 172.21.15.131:0/4046130138' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:44:22.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:21 smithi131 ceph-mon[29507]: from='mgr.14100 172.21.15.131:0/4046130138' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:44:22.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:21 smithi131 ceph-mon[29507]: from='mgr.14100 172.21.15.131:0/4046130138' entity='mgr.y' 2024-02-05T07:44:22.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:21 smithi131 ceph-mon[29507]: from='mgr.14100 172.21.15.131:0/4046130138' entity='mgr.y' 2024-02-05T07:44:22.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:21 smithi131 ceph-mon[29507]: from='mgr.14100 172.21.15.131:0/4046130138' entity='mgr.y' 2024-02-05T07:44:22.367 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-02-05T07:44:22.367 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-02-05T07:44:22.367 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "4018fbe8-c3fa-11ee-95b6-87774f69a715", 2024-02-05T07:44:22.368 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-02-05T07:44:22.368 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-05T07:44:22.368 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-05T07:44:22.368 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-05T07:44:22.368 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:22.368 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-05T07:44:22.368 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-05T07:44:22.368 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-02-05T07:44:22.368 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:44:22.368 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-05T07:44:22.368 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-02-05T07:44:22.368 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:44:22.368 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-02-05T07:44:22.368 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-05T07:44:22.369 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:44:22.369 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-05T07:44:22.369 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-05T07:44:22.369 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:22.369 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-05T07:44:22.369 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:44:22.369 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-05T07:44:22.369 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-05T07:44:22.369 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-05T07:44:22.376 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-05T07:44:22.376 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-05T07:44:22.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-05T07:44:22.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:22.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-05T07:44:22.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-05T07:44:22.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-05T07:44:22.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-05T07:44:22.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-05T07:44:22.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-05T07:44:22.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-05T07:44:22.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-05T07:44:22.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-05T07:44:22.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:22.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-05T07:44:22.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:44:22.377 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-05T07:44:22.378 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-05T07:44:22.378 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:22.378 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-05T07:44:22.378 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-05T07:44:22.378 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-05T07:44:22.378 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-05T07:44:22.378 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-05T07:44:22.378 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-05T07:44:22.378 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-02-05T07:44:22.378 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:44:22.378 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T07:44:22.378 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:22.378 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-05T07:44:22.378 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:44:22.378 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-02-05T07:44:07.661401+0000", 2024-02-05T07:44:22.379 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T07:44:22.379 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:22.379 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-05T07:44:22.379 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-02-05T07:44:22.379 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.773+0000 7f5c5d3f8700 1 Processor -- start 2024-02-05T07:44:22.379 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.773+0000 7f5c5d3f8700 1 -- start start 2024-02-05T07:44:22.379 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.773+0000 7f5c5d3f8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c581068d0 0x7f5c58108cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:22.379 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.773+0000 7f5c5d3f8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c58079740 con 0x7f5c581068d0 2024-02-05T07:44:22.379 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.773+0000 7f5c56ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c581068d0 0x7f5c58108cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:22.379 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.773+0000 7f5c56ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c581068d0 0x7f5c58108cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50242/0 (socket says 172.21.15.131:50242) 2024-02-05T07:44:22.379 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.773+0000 7f5c56ffd700 1 -- 172.21.15.131:0/3783421667 learned_addr learned my addr 172.21.15.131:0/3783421667 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:22.379 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.774+0000 7f5c56ffd700 1 -- 172.21.15.131:0/3783421667 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c58109200 con 0x7f5c581068d0 2024-02-05T07:44:22.379 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.774+0000 7f5c56ffd700 1 --2- 172.21.15.131:0/3783421667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c581068d0 0x7f5c58108cc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5c480098a0 tx=0x7f5c48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329efb8dfcaf1845 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:22.379 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.774+0000 7f5c55ffb700 1 -- 172.21.15.131:0/3783421667 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c48004160 con 0x7f5c581068d0 2024-02-05T07:44:22.379 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.774+0000 7f5c55ffb700 1 -- 172.21.15.131:0/3783421667 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f5c480042c0 con 0x7f5c581068d0 2024-02-05T07:44:22.379 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.774+0000 7f5c55ffb700 1 -- 172.21.15.131:0/3783421667 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c480119b0 con 0x7f5c581068d0 2024-02-05T07:44:22.380 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.774+0000 7f5c5d3f8700 1 -- 172.21.15.131:0/3783421667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c581068d0 msgr2=0x7f5c58108cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:22.380 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.774+0000 7f5c5d3f8700 1 --2- 172.21.15.131:0/3783421667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c581068d0 0x7f5c58108cc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5c480098a0 tx=0x7f5c48009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:22.380 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.774+0000 7f5c5d3f8700 1 -- 172.21.15.131:0/3783421667 shutdown_connections 2024-02-05T07:44:22.380 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.774+0000 7f5c5d3f8700 1 --2- 172.21.15.131:0/3783421667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c581068d0 0x7f5c58108cc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:22.380 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.774+0000 7f5c5d3f8700 1 -- 172.21.15.131:0/3783421667 >> 172.21.15.131:0/3783421667 conn(0x7f5c58100300 msgr2=0x7f5c58102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:22.380 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.774+0000 7f5c5d3f8700 1 -- 172.21.15.131:0/3783421667 shutdown_connections 2024-02-05T07:44:22.380 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.775+0000 7f5c5d3f8700 1 -- 172.21.15.131:0/3783421667 wait complete. 2024-02-05T07:44:22.380 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.775+0000 7f5c5d3f8700 1 Processor -- start 2024-02-05T07:44:22.380 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.775+0000 7f5c5d3f8700 1 -- start start 2024-02-05T07:44:22.380 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.776+0000 7f5c5d3f8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c581068d0 0x7f5c58199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:22.380 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.776+0000 7f5c5d3f8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5819a010 con 0x7f5c581068d0 2024-02-05T07:44:22.380 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.776+0000 7f5c56ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c581068d0 0x7f5c58199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:22.380 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.776+0000 7f5c56ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c581068d0 0x7f5c58199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50254/0 (socket says 172.21.15.131:50254) 2024-02-05T07:44:22.381 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.776+0000 7f5c56ffd700 1 -- 172.21.15.131:0/3491681379 learned_addr learned my addr 172.21.15.131:0/3491681379 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:22.381 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.776+0000 7f5c56ffd700 1 -- 172.21.15.131:0/3491681379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c48009550 con 0x7f5c581068d0 2024-02-05T07:44:22.381 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.776+0000 7f5c56ffd700 1 --2- 172.21.15.131:0/3491681379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c581068d0 0x7f5c58199ad0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5c480046c0 tx=0x7f5c48011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:22.381 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.777+0000 7f5c3ffff700 1 -- 172.21.15.131:0/3491681379 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c480038b0 con 0x7f5c581068d0 2024-02-05T07:44:22.381 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.777+0000 7f5c5d3f8700 1 -- 172.21.15.131:0/3491681379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5819a210 con 0x7f5c581068d0 2024-02-05T07:44:22.381 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.777+0000 7f5c3ffff700 1 -- 172.21.15.131:0/3491681379 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f5c48003a10 con 0x7f5c581068d0 2024-02-05T07:44:22.381 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.777+0000 7f5c3ffff700 1 -- 172.21.15.131:0/3491681379 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c48019690 con 0x7f5c581068d0 2024-02-05T07:44:22.381 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.777+0000 7f5c5d3f8700 1 -- 172.21.15.131:0/3491681379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5819a6b0 con 0x7f5c581068d0 2024-02-05T07:44:22.381 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.778+0000 7f5c3ffff700 1 -- 172.21.15.131:0/3491681379 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 2) v1 ==== 45688+0+0 (secure 0 0 0) 0x7f5c480198c0 con 0x7f5c581068d0 2024-02-05T07:44:22.381 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.778+0000 7f5c5d3f8700 1 -- 172.21.15.131:0/3491681379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c58109f20 con 0x7f5c581068d0 2024-02-05T07:44:22.381 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.778+0000 7f5c3ffff700 1 -- 172.21.15.131:0/3491681379 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5c4804d060 con 0x7f5c581068d0 2024-02-05T07:44:22.381 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.780+0000 7f5c3ffff700 1 -- 172.21.15.131:0/3491681379 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5c48023e00 con 0x7f5c581068d0 2024-02-05T07:44:22.381 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.849+0000 7f5c5d3f8700 1 -- 172.21.15.131:0/3491681379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f5c58104840 con 0x7f5c581068d0 2024-02-05T07:44:22.381 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.850+0000 7f5c3ffff700 1 -- 172.21.15.131:0/3491681379 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f5c480314f0 con 0x7f5c581068d0 2024-02-05T07:44:22.381 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.850+0000 7f5c5d3f8700 1 -- 172.21.15.131:0/3491681379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c581068d0 msgr2=0x7f5c58199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:22.382 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.850+0000 7f5c5d3f8700 1 --2- 172.21.15.131:0/3491681379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c581068d0 0x7f5c58199ad0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5c480046c0 tx=0x7f5c48011a50 comp rx=0 tx=0).stop 2024-02-05T07:44:22.382 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.850+0000 7f5c5d3f8700 1 -- 172.21.15.131:0/3491681379 shutdown_connections 2024-02-05T07:44:22.382 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.850+0000 7f5c5d3f8700 1 --2- 172.21.15.131:0/3491681379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c581068d0 0x7f5c58199ad0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:22.382 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.850+0000 7f5c5d3f8700 1 -- 172.21.15.131:0/3491681379 >> 172.21.15.131:0/3491681379 conn(0x7f5c58100300 msgr2=0x7f5c58100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:22.382 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.851+0000 7f5c5d3f8700 1 -- 172.21.15.131:0/3491681379 shutdown_connections 2024-02-05T07:44:22.382 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:21.851+0000 7f5c5d3f8700 1 -- 172.21.15.131:0/3491681379 wait complete. 2024-02-05T07:44:22.382 INFO:teuthology.orchestra.run.smithi131.stdout:mgr not available, waiting (3/15)... 2024-02-05T07:44:23.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3491681379' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-05T07:44:23.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:22 smithi131 ceph-mon[29507]: mgrmap e3: y(active, since 1.00839s) 2024-02-05T07:44:25.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:24 smithi131 ceph-mon[29507]: mgrmap e4: y(active, since 2s) 2024-02-05T07:44:26.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/97340685' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-05T07:44:31.023 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-02-05T07:44:31.023 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-02-05T07:44:31.023 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "4018fbe8-c3fa-11ee-95b6-87774f69a715", 2024-02-05T07:44:31.023 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-02-05T07:44:31.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-05T07:44:31.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-05T07:44:31.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-05T07:44:31.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:31.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-05T07:44:31.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-05T07:44:31.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-02-05T07:44:31.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:44:31.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-05T07:44:31.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-02-05T07:44:31.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:44:31.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-02-05T07:44:31.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-05T07:44:31.024 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:44:31.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-05T07:44:31.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-05T07:44:31.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:31.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-05T07:44:31.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:44:31.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-05T07:44:31.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-05T07:44:31.025 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-05T07:44:31.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-05T07:44:31.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-05T07:44:31.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-05T07:44:31.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:31.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-05T07:44:31.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-05T07:44:31.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-05T07:44:31.033 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-05T07:44:31.034 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-05T07:44:31.035 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-02-05T07:44:31.035 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:44:31.035 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T07:44:31.035 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:31.035 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-05T07:44:31.035 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:44:31.035 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-02-05T07:44:07.661401+0000", 2024-02-05T07:44:31.035 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T07:44:31.035 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:44:31.035 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-05T07:44:31.035 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-02-05T07:44:31.035 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.854+0000 7f1ec1f71700 1 Processor -- start 2024-02-05T07:44:31.035 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.854+0000 7f1ec1f71700 1 -- start start 2024-02-05T07:44:31.035 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.854+0000 7f1ec1f71700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ebc104db0 0x7f1ebc1051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:31.035 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.854+0000 7f1ec1f71700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc1057a0 con 0x7f1ebc104db0 2024-02-05T07:44:31.035 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.855+0000 7f1ebb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ebc104db0 0x7f1ebc1051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:31.036 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.855+0000 7f1ebb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ebc104db0 0x7f1ebc1051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50268/0 (socket says 172.21.15.131:50268) 2024-02-05T07:44:31.036 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.855+0000 7f1ebb7fe700 1 -- 172.21.15.131:0/235762149 learned_addr learned my addr 172.21.15.131:0/235762149 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:31.036 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.855+0000 7f1ebb7fe700 1 -- 172.21.15.131:0/235762149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc105fb0 con 0x7f1ebc104db0 2024-02-05T07:44:31.036 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.855+0000 7f1ebb7fe700 1 --2- 172.21.15.131:0/235762149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ebc104db0 0x7f1ebc1051d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1eac0098a0 tx=0x7f1eac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04598bf914bee73 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:31.036 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.855+0000 7f1ebaffd700 1 -- 172.21.15.131:0/235762149 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1eac004160 con 0x7f1ebc104db0 2024-02-05T07:44:31.036 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.855+0000 7f1ebaffd700 1 -- 172.21.15.131:0/235762149 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f1eac0042c0 con 0x7f1ebc104db0 2024-02-05T07:44:31.036 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.855+0000 7f1ebaffd700 1 -- 172.21.15.131:0/235762149 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1eac0119b0 con 0x7f1ebc104db0 2024-02-05T07:44:31.036 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.856+0000 7f1ec1f71700 1 -- 172.21.15.131:0/235762149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ebc104db0 msgr2=0x7f1ebc1051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:31.036 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.856+0000 7f1ec1f71700 1 --2- 172.21.15.131:0/235762149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ebc104db0 0x7f1ebc1051d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1eac0098a0 tx=0x7f1eac009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:31.036 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.856+0000 7f1ec1f71700 1 -- 172.21.15.131:0/235762149 shutdown_connections 2024-02-05T07:44:31.036 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.856+0000 7f1ec1f71700 1 --2- 172.21.15.131:0/235762149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ebc104db0 0x7f1ebc1051d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:31.036 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.856+0000 7f1ec1f71700 1 -- 172.21.15.131:0/235762149 >> 172.21.15.131:0/235762149 conn(0x7f1ebc1003b0 msgr2=0x7f1ebc1027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:31.036 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.856+0000 7f1ec1f71700 1 -- 172.21.15.131:0/235762149 shutdown_connections 2024-02-05T07:44:31.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.856+0000 7f1ec1f71700 1 -- 172.21.15.131:0/235762149 wait complete. 2024-02-05T07:44:31.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.857+0000 7f1ec1f71700 1 Processor -- start 2024-02-05T07:44:31.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.857+0000 7f1ec1f71700 1 -- start start 2024-02-05T07:44:31.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.857+0000 7f1ec1f71700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ebc104db0 0x7f1ebc199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:31.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.857+0000 7f1ec1f71700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc19a090 con 0x7f1ebc104db0 2024-02-05T07:44:31.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.857+0000 7f1ebb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ebc104db0 0x7f1ebc199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:31.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.857+0000 7f1ebb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ebc104db0 0x7f1ebc199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50274/0 (socket says 172.21.15.131:50274) 2024-02-05T07:44:31.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.857+0000 7f1ebb7fe700 1 -- 172.21.15.131:0/97340685 learned_addr learned my addr 172.21.15.131:0/97340685 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:31.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.857+0000 7f1ebb7fe700 1 -- 172.21.15.131:0/97340685 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eac009550 con 0x7f1ebc104db0 2024-02-05T07:44:31.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.857+0000 7f1ebb7fe700 1 --2- 172.21.15.131:0/97340685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ebc104db0 0x7f1ebc199b50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1eac006af0 tx=0x7f1eac011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:31.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.857+0000 7f1eb97fa700 1 -- 172.21.15.131:0/97340685 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1eac0039e0 con 0x7f1ebc104db0 2024-02-05T07:44:31.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.858+0000 7f1eb97fa700 1 -- 172.21.15.131:0/97340685 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f1eac003b40 con 0x7f1ebc104db0 2024-02-05T07:44:31.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.858+0000 7f1ec1f71700 1 -- 172.21.15.131:0/97340685 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc19a290 con 0x7f1ebc104db0 2024-02-05T07:44:31.037 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.858+0000 7f1eb97fa700 1 -- 172.21.15.131:0/97340685 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1eac019700 con 0x7f1ebc104db0 2024-02-05T07:44:31.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.858+0000 7f1ec1f71700 1 -- 172.21.15.131:0/97340685 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc19a730 con 0x7f1ebc104db0 2024-02-05T07:44:31.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.858+0000 7f1eb97fa700 1 -- 172.21.15.131:0/97340685 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) v1 ==== 45886+0+0 (secure 0 0 0) 0x7f1eac019960 con 0x7f1ebc104db0 2024-02-05T07:44:31.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.858+0000 7f1ec1f71700 1 -- 172.21.15.131:0/97340685 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc193c80 con 0x7f1ebc104db0 2024-02-05T07:44:31.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.859+0000 7f1eb97fa700 1 --2- 172.21.15.131:0/97340685 >> v2:172.21.15.131:6800/1416975616 conn(0x7f1ea8038ea0 0x7f1ea803b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:31.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.859+0000 7f1eb97fa700 1 -- 172.21.15.131:0/97340685 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1eac04cfe0 con 0x7f1ebc104db0 2024-02-05T07:44:31.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.859+0000 7f1eb3fff700 1 --2- 172.21.15.131:0/97340685 >> v2:172.21.15.131:6800/1416975616 conn(0x7f1ea8038ea0 0x7f1ea803b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:31.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.859+0000 7f1eb3fff700 1 --2- 172.21.15.131:0/97340685 >> v2:172.21.15.131:6800/1416975616 conn(0x7f1ea8038ea0 0x7f1ea803b360 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4006fd0 tx=0x7f1ea4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:31.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:24.864+0000 7f1eb97fa700 1 -- 172.21.15.131:0/97340685 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1eac010c50 con 0x7f1ebc104db0 2024-02-05T07:44:31.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:25.135+0000 7f1ec1f71700 1 -- 172.21.15.131:0/97340685 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f1ebc063350 con 0x7f1ebc104db0 2024-02-05T07:44:31.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:25.136+0000 7f1eb97fa700 1 -- 172.21.15.131:0/97340685 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f1eac010380 con 0x7f1ebc104db0 2024-02-05T07:44:31.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:25.138+0000 7f1ec1f71700 1 -- 172.21.15.131:0/97340685 >> v2:172.21.15.131:6800/1416975616 conn(0x7f1ea8038ea0 msgr2=0x7f1ea803b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:31.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:25.138+0000 7f1ec1f71700 1 --2- 172.21.15.131:0/97340685 >> v2:172.21.15.131:6800/1416975616 conn(0x7f1ea8038ea0 0x7f1ea803b360 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4006fd0 tx=0x7f1ea4006e40 comp rx=0 tx=0).stop 2024-02-05T07:44:31.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:25.138+0000 7f1ec1f71700 1 -- 172.21.15.131:0/97340685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ebc104db0 msgr2=0x7f1ebc199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:31.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:25.138+0000 7f1ec1f71700 1 --2- 172.21.15.131:0/97340685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ebc104db0 0x7f1ebc199b50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1eac006af0 tx=0x7f1eac011a50 comp rx=0 tx=0).stop 2024-02-05T07:44:31.038 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:25.138+0000 7f1ec1f71700 1 -- 172.21.15.131:0/97340685 shutdown_connections 2024-02-05T07:44:31.039 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:25.138+0000 7f1ec1f71700 1 --2- 172.21.15.131:0/97340685 >> v2:172.21.15.131:6800/1416975616 conn(0x7f1ea8038ea0 0x7f1ea803b360 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:31.039 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:25.138+0000 7f1ec1f71700 1 --2- 172.21.15.131:0/97340685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ebc104db0 0x7f1ebc199b50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:31.039 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:25.138+0000 7f1ec1f71700 1 -- 172.21.15.131:0/97340685 >> 172.21.15.131:0/97340685 conn(0x7f1ebc1003b0 msgr2=0x7f1ebc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:31.039 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:25.138+0000 7f1ec1f71700 1 -- 172.21.15.131:0/97340685 shutdown_connections 2024-02-05T07:44:31.039 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:25.139+0000 7f1ec1f71700 1 -- 172.21.15.131:0/97340685 wait complete. 2024-02-05T07:44:31.039 INFO:teuthology.orchestra.run.smithi131.stdout:mgr is available 2024-02-05T07:44:34.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/399439666' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-05T07:44:35.046 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-02-05T07:44:35.046 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [global] 2024-02-05T07:44:35.046 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout fsid = 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:44:35.046 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-05T07:44:35.046 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-05T07:44:35.047 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-05T07:44:35.047 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-05T07:44:35.047 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-02-05T07:44:35.047 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-05T07:44:35.047 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-05T07:44:35.047 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-02-05T07:44:35.047 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [osd] 2024-02-05T07:44:35.047 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-05T07:44:35.047 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-05T07:44:35.047 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.269+0000 7f561cac4700 1 Processor -- start 2024-02-05T07:44:35.047 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.269+0000 7f561cac4700 1 -- start start 2024-02-05T07:44:35.048 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.269+0000 7f561cac4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5618105e60 0x7f5618106280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:35.048 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.269+0000 7f561cac4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56181067c0 con 0x7f5618105e60 2024-02-05T07:44:35.048 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.269+0000 7f561659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5618105e60 0x7f5618106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:35.048 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.269+0000 7f561659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5618105e60 0x7f5618106280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60922/0 (socket says 172.21.15.131:60922) 2024-02-05T07:44:35.048 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.269+0000 7f561659c700 1 -- 172.21.15.131:0/3097714743 learned_addr learned my addr 172.21.15.131:0/3097714743 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:35.048 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.270+0000 7f561659c700 1 -- 172.21.15.131:0/3097714743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5618106900 con 0x7f5618105e60 2024-02-05T07:44:35.048 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.270+0000 7f561659c700 1 --2- 172.21.15.131:0/3097714743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5618105e60 0x7f5618106280 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f56080098a0 tx=0x7f5608009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8b9005b70d2e37 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:35.048 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.270+0000 7f561559a700 1 -- 172.21.15.131:0/3097714743 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5608004160 con 0x7f5618105e60 2024-02-05T07:44:35.048 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.270+0000 7f561559a700 1 -- 172.21.15.131:0/3097714743 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f56080042c0 con 0x7f5618105e60 2024-02-05T07:44:35.048 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.270+0000 7f561559a700 1 -- 172.21.15.131:0/3097714743 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56080119b0 con 0x7f5618105e60 2024-02-05T07:44:35.048 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.271+0000 7f561cac4700 1 -- 172.21.15.131:0/3097714743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5618105e60 msgr2=0x7f5618106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:35.048 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.271+0000 7f561cac4700 1 --2- 172.21.15.131:0/3097714743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5618105e60 0x7f5618106280 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f56080098a0 tx=0x7f5608009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:35.049 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.271+0000 7f561cac4700 1 -- 172.21.15.131:0/3097714743 shutdown_connections 2024-02-05T07:44:35.049 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.271+0000 7f561cac4700 1 --2- 172.21.15.131:0/3097714743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5618105e60 0x7f5618106280 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:35.049 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.271+0000 7f561cac4700 1 -- 172.21.15.131:0/3097714743 >> 172.21.15.131:0/3097714743 conn(0x7f56181014d0 msgr2=0x7f56181038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:35.049 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.271+0000 7f561cac4700 1 -- 172.21.15.131:0/3097714743 shutdown_connections 2024-02-05T07:44:35.049 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.271+0000 7f561cac4700 1 -- 172.21.15.131:0/3097714743 wait complete. 2024-02-05T07:44:35.049 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.271+0000 7f561cac4700 1 Processor -- start 2024-02-05T07:44:35.049 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.271+0000 7f561cac4700 1 -- start start 2024-02-05T07:44:35.049 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.272+0000 7f561cac4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5618105e60 0x7f5618199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:35.049 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.272+0000 7f561cac4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561819a050 con 0x7f5618105e60 2024-02-05T07:44:35.049 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.272+0000 7f561659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5618105e60 0x7f5618199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:35.049 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.272+0000 7f561659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5618105e60 0x7f5618199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60932/0 (socket says 172.21.15.131:60932) 2024-02-05T07:44:35.049 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.272+0000 7f561659c700 1 -- 172.21.15.131:0/399439666 learned_addr learned my addr 172.21.15.131:0/399439666 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:35.049 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.272+0000 7f561659c700 1 -- 172.21.15.131:0/399439666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5608009550 con 0x7f5618105e60 2024-02-05T07:44:35.050 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.272+0000 7f561659c700 1 --2- 172.21.15.131:0/399439666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5618105e60 0x7f5618199b10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5608006af0 tx=0x7f5608011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:35.050 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.272+0000 7f56077fe700 1 -- 172.21.15.131:0/399439666 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56080039e0 con 0x7f5618105e60 2024-02-05T07:44:35.050 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.272+0000 7f56077fe700 1 -- 172.21.15.131:0/399439666 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f5608003b40 con 0x7f5618105e60 2024-02-05T07:44:35.050 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.272+0000 7f561cac4700 1 -- 172.21.15.131:0/399439666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561819a250 con 0x7f5618105e60 2024-02-05T07:44:35.050 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.272+0000 7f56077fe700 1 -- 172.21.15.131:0/399439666 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5608019700 con 0x7f5618105e60 2024-02-05T07:44:35.050 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.272+0000 7f561cac4700 1 -- 172.21.15.131:0/399439666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561819a6f0 con 0x7f5618105e60 2024-02-05T07:44:35.050 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.273+0000 7f56077fe700 1 -- 172.21.15.131:0/399439666 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) v1 ==== 45886+0+0 (secure 0 0 0) 0x7f5608019860 con 0x7f5618105e60 2024-02-05T07:44:35.050 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.273+0000 7f56077fe700 1 --2- 172.21.15.131:0/399439666 >> v2:172.21.15.131:6800/1416975616 conn(0x7f5600038ea0 0x7f560003b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:35.050 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.273+0000 7f56077fe700 1 -- 172.21.15.131:0/399439666 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f560804cfc0 con 0x7f5618105e60 2024-02-05T07:44:35.050 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.273+0000 7f561cac4700 1 -- 172.21.15.131:0/399439666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5618063350 con 0x7f5618105e60 2024-02-05T07:44:35.050 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.274+0000 7f5615d9b700 1 --2- 172.21.15.131:0/399439666 >> v2:172.21.15.131:6800/1416975616 conn(0x7f5600038ea0 0x7f560003b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:35.050 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.274+0000 7f5615d9b700 1 --2- 172.21.15.131:0/399439666 >> v2:172.21.15.131:6800/1416975616 conn(0x7f5600038ea0 0x7f560003b360 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f560c006fd0 tx=0x7f560c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:35.051 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.278+0000 7f56077fe700 1 -- 172.21.15.131:0/399439666 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5608019b10 con 0x7f5618105e60 2024-02-05T07:44:35.051 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.471+0000 7f561cac4700 1 -- 172.21.15.131:0/399439666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f5618107bf0 con 0x7f5618105e60 2024-02-05T07:44:35.051 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.471+0000 7f56077fe700 1 -- 172.21.15.131:0/399439666 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+282 (secure 0 0 0) 0x7f5608010380 con 0x7f5618105e60 2024-02-05T07:44:35.051 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.474+0000 7f561cac4700 1 -- 172.21.15.131:0/399439666 >> v2:172.21.15.131:6800/1416975616 conn(0x7f5600038ea0 msgr2=0x7f560003b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:35.051 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.474+0000 7f561cac4700 1 --2- 172.21.15.131:0/399439666 >> v2:172.21.15.131:6800/1416975616 conn(0x7f5600038ea0 0x7f560003b360 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f560c006fd0 tx=0x7f560c006e40 comp rx=0 tx=0).stop 2024-02-05T07:44:35.051 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.474+0000 7f561cac4700 1 -- 172.21.15.131:0/399439666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5618105e60 msgr2=0x7f5618199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:35.051 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.474+0000 7f561cac4700 1 --2- 172.21.15.131:0/399439666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5618105e60 0x7f5618199b10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5608006af0 tx=0x7f5608011a50 comp rx=0 tx=0).stop 2024-02-05T07:44:35.051 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.474+0000 7f561cac4700 1 -- 172.21.15.131:0/399439666 shutdown_connections 2024-02-05T07:44:35.051 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.474+0000 7f561cac4700 1 --2- 172.21.15.131:0/399439666 >> v2:172.21.15.131:6800/1416975616 conn(0x7f5600038ea0 0x7f560003b360 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:35.051 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.474+0000 7f561cac4700 1 --2- 172.21.15.131:0/399439666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5618105e60 0x7f5618199b10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:35.051 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.474+0000 7f561cac4700 1 -- 172.21.15.131:0/399439666 >> 172.21.15.131:0/399439666 conn(0x7f56181014d0 msgr2=0x7f56181020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:35.051 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.474+0000 7f561cac4700 1 -- 172.21.15.131:0/399439666 shutdown_connections 2024-02-05T07:44:35.051 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:34.474+0000 7f561cac4700 1 -- 172.21.15.131:0/399439666 wait complete. 2024-02-05T07:44:35.051 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling cephadm module... 2024-02-05T07:44:36.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4282920266' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-05T07:44:36.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:35 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ignoring --setuser ceph since I am not root 2024-02-05T07:44:36.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:35 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ignoring --setgroup ceph since I am not root 2024-02-05T07:44:36.767 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:36 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:36.360+0000 7fa6d792d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T07:44:36.768 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:36 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:36.453+0000 7fa6d792d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T07:44:37.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4282920266' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-05T07:44:37.068 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:36 smithi131 ceph-mon[29507]: mgrmap e5: y(active, since 14s) 2024-02-05T07:44:37.075 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.506+0000 7f643c4e9700 1 Processor -- start 2024-02-05T07:44:37.075 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.506+0000 7f643c4e9700 1 -- start start 2024-02-05T07:44:37.075 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.507+0000 7f643c4e9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434104d00 0x7f6434105120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:37.075 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.507+0000 7f643c4e9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64341056f0 con 0x7f6434104d00 2024-02-05T07:44:37.075 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.507+0000 7f643a285700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434104d00 0x7f6434105120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:37.076 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.507+0000 7f643a285700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434104d00 0x7f6434105120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60948/0 (socket says 172.21.15.131:60948) 2024-02-05T07:44:37.076 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.507+0000 7f643a285700 1 -- 172.21.15.131:0/118394439 learned_addr learned my addr 172.21.15.131:0/118394439 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:37.076 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.507+0000 7f643a285700 1 -- 172.21.15.131:0/118394439 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6434105f00 con 0x7f6434104d00 2024-02-05T07:44:37.076 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.507+0000 7f643a285700 1 --2- 172.21.15.131:0/118394439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434104d00 0x7f6434105120 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f64240098a0 tx=0x7f6424009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6edcb344fbc07a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:37.076 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.507+0000 7f6439283700 1 -- 172.21.15.131:0/118394439 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6424004160 con 0x7f6434104d00 2024-02-05T07:44:37.076 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.507+0000 7f6439283700 1 -- 172.21.15.131:0/118394439 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f64240042c0 con 0x7f6434104d00 2024-02-05T07:44:37.076 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.507+0000 7f6439283700 1 -- 172.21.15.131:0/118394439 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64240119b0 con 0x7f6434104d00 2024-02-05T07:44:37.076 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.508+0000 7f643c4e9700 1 -- 172.21.15.131:0/118394439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434104d00 msgr2=0x7f6434105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:37.076 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.508+0000 7f643c4e9700 1 --2- 172.21.15.131:0/118394439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434104d00 0x7f6434105120 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f64240098a0 tx=0x7f6424009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:37.076 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.508+0000 7f643c4e9700 1 -- 172.21.15.131:0/118394439 shutdown_connections 2024-02-05T07:44:37.076 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.508+0000 7f643c4e9700 1 --2- 172.21.15.131:0/118394439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434104d00 0x7f6434105120 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:37.076 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.508+0000 7f643c4e9700 1 -- 172.21.15.131:0/118394439 >> 172.21.15.131:0/118394439 conn(0x7f64341002c0 msgr2=0x7f64341026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:37.076 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.508+0000 7f643c4e9700 1 -- 172.21.15.131:0/118394439 shutdown_connections 2024-02-05T07:44:37.076 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.508+0000 7f643c4e9700 1 -- 172.21.15.131:0/118394439 wait complete. 2024-02-05T07:44:37.076 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.509+0000 7f643c4e9700 1 Processor -- start 2024-02-05T07:44:37.077 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.509+0000 7f643c4e9700 1 -- start start 2024-02-05T07:44:37.077 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.509+0000 7f643c4e9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434104d00 0x7f6434198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:37.077 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.509+0000 7f643c4e9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6434198ec0 con 0x7f6434104d00 2024-02-05T07:44:37.077 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.509+0000 7f643a285700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434104d00 0x7f6434198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:37.077 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.509+0000 7f643a285700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434104d00 0x7f6434198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60954/0 (socket says 172.21.15.131:60954) 2024-02-05T07:44:37.077 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.509+0000 7f643a285700 1 -- 172.21.15.131:0/4282920266 learned_addr learned my addr 172.21.15.131:0/4282920266 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:37.077 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.509+0000 7f643a285700 1 -- 172.21.15.131:0/4282920266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6424009550 con 0x7f6434104d00 2024-02-05T07:44:37.077 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.509+0000 7f643a285700 1 --2- 172.21.15.131:0/4282920266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434104d00 0x7f6434198980 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6424006af0 tx=0x7f6424011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:37.077 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.509+0000 7f642b7fe700 1 -- 172.21.15.131:0/4282920266 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64240039e0 con 0x7f6434104d00 2024-02-05T07:44:37.077 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.509+0000 7f642b7fe700 1 -- 172.21.15.131:0/4282920266 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f6424003b40 con 0x7f6434104d00 2024-02-05T07:44:37.077 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.509+0000 7f643c4e9700 1 -- 172.21.15.131:0/4282920266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64341990c0 con 0x7f6434104d00 2024-02-05T07:44:37.077 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.510+0000 7f642b7fe700 1 -- 172.21.15.131:0/4282920266 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6424019720 con 0x7f6434104d00 2024-02-05T07:44:37.077 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.510+0000 7f643c4e9700 1 -- 172.21.15.131:0/4282920266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6434199560 con 0x7f6434104d00 2024-02-05T07:44:37.077 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.510+0000 7f642b7fe700 1 -- 172.21.15.131:0/4282920266 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) v1 ==== 45886+0+0 (secure 0 0 0) 0x7f642401aba0 con 0x7f6434104d00 2024-02-05T07:44:37.077 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.511+0000 7f642b7fe700 1 --2- 172.21.15.131:0/4282920266 >> v2:172.21.15.131:6800/1416975616 conn(0x7f6420038ea0 0x7f642003b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:37.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.510+0000 7f643c4e9700 1 -- 172.21.15.131:0/4282920266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6434063350 con 0x7f6434104d00 2024-02-05T07:44:37.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.511+0000 7f642b7fe700 1 -- 172.21.15.131:0/4282920266 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f642404ce30 con 0x7f6434104d00 2024-02-05T07:44:37.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.511+0000 7f6439a84700 1 --2- 172.21.15.131:0/4282920266 >> v2:172.21.15.131:6800/1416975616 conn(0x7f6420038ea0 0x7f642003b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:37.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.511+0000 7f6439a84700 1 --2- 172.21.15.131:0/4282920266 >> v2:172.21.15.131:6800/1416975616 conn(0x7f6420038ea0 0x7f642003b360 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6430006fd0 tx=0x7f6430006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:37.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.515+0000 7f642b7fe700 1 -- 172.21.15.131:0/4282920266 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6424010740 con 0x7f6434104d00 2024-02-05T07:44:37.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.760+0000 7f643c4e9700 1 -- 172.21.15.131:0/4282920266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f643419c020 con 0x7f6434104d00 2024-02-05T07:44:37.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.811+0000 7f642b7fe700 1 -- 172.21.15.131:0/4282920266 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f6424010380 con 0x7f6434104d00 2024-02-05T07:44:37.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.813+0000 7f642b7fe700 1 -- 172.21.15.131:0/4282920266 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mgrmap(e 5) v1 ==== 45897+0+0 (secure 0 0 0) 0x7f642402c3a0 con 0x7f6434104d00 2024-02-05T07:44:37.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.815+0000 7f643c4e9700 1 -- 172.21.15.131:0/4282920266 >> v2:172.21.15.131:6800/1416975616 conn(0x7f6420038ea0 msgr2=0x7f642003b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:37.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.815+0000 7f643c4e9700 1 --2- 172.21.15.131:0/4282920266 >> v2:172.21.15.131:6800/1416975616 conn(0x7f6420038ea0 0x7f642003b360 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6430006fd0 tx=0x7f6430006e40 comp rx=0 tx=0).stop 2024-02-05T07:44:37.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.815+0000 7f643c4e9700 1 -- 172.21.15.131:0/4282920266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434104d00 msgr2=0x7f6434198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:37.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.815+0000 7f643c4e9700 1 --2- 172.21.15.131:0/4282920266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434104d00 0x7f6434198980 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6424006af0 tx=0x7f6424011a50 comp rx=0 tx=0).stop 2024-02-05T07:44:37.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.815+0000 7f643c4e9700 1 -- 172.21.15.131:0/4282920266 shutdown_connections 2024-02-05T07:44:37.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.815+0000 7f643c4e9700 1 --2- 172.21.15.131:0/4282920266 >> v2:172.21.15.131:6800/1416975616 conn(0x7f6420038ea0 0x7f642003b360 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:37.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.815+0000 7f643c4e9700 1 --2- 172.21.15.131:0/4282920266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6434104d00 0x7f6434198980 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:37.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.815+0000 7f643c4e9700 1 -- 172.21.15.131:0/4282920266 >> 172.21.15.131:0/4282920266 conn(0x7f64341002c0 msgr2=0x7f6434100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:37.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.815+0000 7f643c4e9700 1 -- 172.21.15.131:0/4282920266 shutdown_connections 2024-02-05T07:44:37.079 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:35.815+0000 7f643c4e9700 1 -- 172.21.15.131:0/4282920266 wait complete. 2024-02-05T07:44:37.324 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:37 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:37.093+0000 7fa6d792d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T07:44:37.324 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:37 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:37.188+0000 7fa6d792d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T07:44:37.753 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:37 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:37.335+0000 7fa6d792d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T07:44:37.753 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:37 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:37.432+0000 7fa6d792d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T07:44:37.753 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:37 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:37.519+0000 7fa6d792d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T07:44:38.017 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:37 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:37.798+0000 7fa6d792d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T07:44:38.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:37 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:37.911+0000 7fa6d792d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T07:44:38.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:37 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:37.997+0000 7fa6d792d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T07:44:38.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-02-05T07:44:38.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-05T07:44:38.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-05T07:44:38.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-05T07:44:38.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-05T07:44:38.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-02-05T07:44:38.350 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.546+0000 7fcb3ce37700 1 Processor -- start 2024-02-05T07:44:38.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.547+0000 7fcb3ce37700 1 -- start start 2024-02-05T07:44:38.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.547+0000 7fcb3ce37700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb38104d10 0x7fcb38105130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:38.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.547+0000 7fcb3ce37700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb38105700 con 0x7fcb38104d10 2024-02-05T07:44:38.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.547+0000 7fcb3659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb38104d10 0x7fcb38105130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:38.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.547+0000 7fcb3659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb38104d10 0x7fcb38105130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60982/0 (socket says 172.21.15.131:60982) 2024-02-05T07:44:38.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.547+0000 7fcb3659c700 1 -- 172.21.15.131:0/799758332 learned_addr learned my addr 172.21.15.131:0/799758332 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:38.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.547+0000 7fcb3659c700 1 -- 172.21.15.131:0/799758332 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb38105f10 con 0x7fcb38104d10 2024-02-05T07:44:38.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.548+0000 7fcb3659c700 1 --2- 172.21.15.131:0/799758332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb38104d10 0x7fcb38105130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcb280098a0 tx=0x7fcb28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33126b933060092d server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:38.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.548+0000 7fcb3559a700 1 -- 172.21.15.131:0/799758332 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb28004160 con 0x7fcb38104d10 2024-02-05T07:44:38.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.548+0000 7fcb3559a700 1 -- 172.21.15.131:0/799758332 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fcb280042c0 con 0x7fcb38104d10 2024-02-05T07:44:38.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.548+0000 7fcb3559a700 1 -- 172.21.15.131:0/799758332 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb280119b0 con 0x7fcb38104d10 2024-02-05T07:44:38.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.548+0000 7fcb3ce37700 1 -- 172.21.15.131:0/799758332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb38104d10 msgr2=0x7fcb38105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:38.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.548+0000 7fcb3ce37700 1 --2- 172.21.15.131:0/799758332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb38104d10 0x7fcb38105130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcb280098a0 tx=0x7fcb28009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:38.351 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.548+0000 7fcb3ce37700 1 -- 172.21.15.131:0/799758332 shutdown_connections 2024-02-05T07:44:38.352 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.548+0000 7fcb3ce37700 1 --2- 172.21.15.131:0/799758332 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb38104d10 0x7fcb38105130 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:38.352 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.548+0000 7fcb3ce37700 1 -- 172.21.15.131:0/799758332 >> 172.21.15.131:0/799758332 conn(0x7fcb38100270 msgr2=0x7fcb381026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:38.352 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.549+0000 7fcb3ce37700 1 -- 172.21.15.131:0/799758332 shutdown_connections 2024-02-05T07:44:38.352 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.549+0000 7fcb3ce37700 1 -- 172.21.15.131:0/799758332 wait complete. 2024-02-05T07:44:38.352 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.549+0000 7fcb3ce37700 1 Processor -- start 2024-02-05T07:44:38.352 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.549+0000 7fcb3ce37700 1 -- start start 2024-02-05T07:44:38.352 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.549+0000 7fcb3ce37700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb38104d10 0x7fcb38199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:38.352 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.549+0000 7fcb3ce37700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3819a000 con 0x7fcb38104d10 2024-02-05T07:44:38.352 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.550+0000 7fcb3659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb38104d10 0x7fcb38199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:38.352 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.550+0000 7fcb3659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb38104d10 0x7fcb38199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60998/0 (socket says 172.21.15.131:60998) 2024-02-05T07:44:38.352 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.550+0000 7fcb3659c700 1 -- 172.21.15.131:0/1278591199 learned_addr learned my addr 172.21.15.131:0/1278591199 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:38.352 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.550+0000 7fcb3659c700 1 -- 172.21.15.131:0/1278591199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb28009550 con 0x7fcb38104d10 2024-02-05T07:44:38.352 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.550+0000 7fcb3659c700 1 --2- 172.21.15.131:0/1278591199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb38104d10 0x7fcb38199ac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcb28006af0 tx=0x7fcb28011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:38.352 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.550+0000 7fcb277fe700 1 -- 172.21.15.131:0/1278591199 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb280039e0 con 0x7fcb38104d10 2024-02-05T07:44:38.353 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.550+0000 7fcb277fe700 1 -- 172.21.15.131:0/1278591199 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fcb28003b40 con 0x7fcb38104d10 2024-02-05T07:44:38.353 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.550+0000 7fcb3ce37700 1 -- 172.21.15.131:0/1278591199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3819a200 con 0x7fcb38104d10 2024-02-05T07:44:38.353 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.550+0000 7fcb277fe700 1 -- 172.21.15.131:0/1278591199 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb28019720 con 0x7fcb38104d10 2024-02-05T07:44:38.353 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.550+0000 7fcb3ce37700 1 -- 172.21.15.131:0/1278591199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3819a6a0 con 0x7fcb38104d10 2024-02-05T07:44:38.353 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.551+0000 7fcb277fe700 1 -- 172.21.15.131:0/1278591199 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 45897+0+0 (secure 0 0 0) 0x7fcb28019970 con 0x7fcb38104d10 2024-02-05T07:44:38.353 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.551+0000 7fcb3ce37700 1 -- 172.21.15.131:0/1278591199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb38193be0 con 0x7fcb38104d10 2024-02-05T07:44:38.353 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.551+0000 7fcb277fe700 1 --2- 172.21.15.131:0/1278591199 >> v2:172.21.15.131:6800/1416975616 conn(0x7fcb20038ef0 0x7fcb2003b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:38.353 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.551+0000 7fcb277fe700 1 -- 172.21.15.131:0/1278591199 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcb2804de70 con 0x7fcb38104d10 2024-02-05T07:44:38.353 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.551+0000 7fcb35d9b700 1 -- 172.21.15.131:0/1278591199 >> v2:172.21.15.131:6800/1416975616 conn(0x7fcb20038ef0 msgr2=0x7fcb2003b3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1416975616 2024-02-05T07:44:38.353 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.551+0000 7fcb35d9b700 1 --2- 172.21.15.131:0/1278591199 >> v2:172.21.15.131:6800/1416975616 conn(0x7fcb20038ef0 0x7fcb2003b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-05T07:44:38.353 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.556+0000 7fcb277fe700 1 -- 172.21.15.131:0/1278591199 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcb28019e00 con 0x7fcb38104d10 2024-02-05T07:44:38.353 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.752+0000 7fcb35d9b700 1 -- 172.21.15.131:0/1278591199 >> v2:172.21.15.131:6800/1416975616 conn(0x7fcb20038ef0 msgr2=0x7fcb2003b3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1416975616 2024-02-05T07:44:38.353 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.752+0000 7fcb35d9b700 1 --2- 172.21.15.131:0/1278591199 >> v2:172.21.15.131:6800/1416975616 conn(0x7fcb20038ef0 0x7fcb2003b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-05T07:44:38.353 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.810+0000 7fcb3ce37700 1 -- 172.21.15.131:0/1278591199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fcb38063350 con 0x7fcb38104d10 2024-02-05T07:44:38.354 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.811+0000 7fcb277fe700 1 -- 172.21.15.131:0/1278591199 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fcb28010380 con 0x7fcb38104d10 2024-02-05T07:44:38.354 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.813+0000 7fcb3ce37700 1 -- 172.21.15.131:0/1278591199 >> v2:172.21.15.131:6800/1416975616 conn(0x7fcb20038ef0 msgr2=0x7fcb2003b3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-05T07:44:38.354 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.813+0000 7fcb3ce37700 1 --2- 172.21.15.131:0/1278591199 >> v2:172.21.15.131:6800/1416975616 conn(0x7fcb20038ef0 0x7fcb2003b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:38.354 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.813+0000 7fcb3ce37700 1 -- 172.21.15.131:0/1278591199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb38104d10 msgr2=0x7fcb38199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:38.354 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.813+0000 7fcb3ce37700 1 --2- 172.21.15.131:0/1278591199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb38104d10 0x7fcb38199ac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcb28006af0 tx=0x7fcb28011a50 comp rx=0 tx=0).stop 2024-02-05T07:44:38.354 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.813+0000 7fcb3ce37700 1 -- 172.21.15.131:0/1278591199 shutdown_connections 2024-02-05T07:44:38.354 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.813+0000 7fcb3ce37700 1 --2- 172.21.15.131:0/1278591199 >> v2:172.21.15.131:6800/1416975616 conn(0x7fcb20038ef0 0x7fcb2003b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:38.354 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.813+0000 7fcb3ce37700 1 --2- 172.21.15.131:0/1278591199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb38104d10 0x7fcb38199ac0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:38.354 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.813+0000 7fcb3ce37700 1 -- 172.21.15.131:0/1278591199 >> 172.21.15.131:0/1278591199 conn(0x7fcb38100270 msgr2=0x7fcb381011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:38.354 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.813+0000 7fcb3ce37700 1 -- 172.21.15.131:0/1278591199 shutdown_connections 2024-02-05T07:44:38.354 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:37.813+0000 7fcb3ce37700 1 -- 172.21.15.131:0/1278591199 wait complete. 2024-02-05T07:44:38.354 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for the mgr to restart... 2024-02-05T07:44:38.354 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr epoch 5... 2024-02-05T07:44:38.457 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:38 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:38.094+0000 7fa6d792d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T07:44:38.458 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:38 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:38.273+0000 7fa6d792d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T07:44:38.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:38 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1278591199' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-05T07:44:39.517 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:39 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:39.248+0000 7fa6d792d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T07:44:39.518 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:39 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:39.378+0000 7fa6d792d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T07:44:39.988 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:39 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:39.707+0000 7fa6d792d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T07:44:39.988 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:39 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:39.796+0000 7fa6d792d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T07:44:39.988 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:39 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:39.987+0000 7fa6d792d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:44:40.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:40 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:40.168+0000 7fa6d792d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T07:44:40.704 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:40 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:40.446+0000 7fa6d792d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:44:40.705 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:40 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:40.703+0000 7fa6d792d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T07:44:41.017 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:40 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:40.936+0000 7fa6d792d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T07:44:42.017 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:41 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:41.715+0000 7fa6d792d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T07:44:42.586 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:42.279+0000 7fa6d792d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T07:44:42.586 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:42.374+0000 7fa6d792d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T07:44:42.586 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:44:42.584+0000 7fa6d792d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T07:44:43.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-mon[29507]: Active manager daemon y restarted 2024-02-05T07:44:43.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-mon[29507]: Activating manager daemon y 2024-02-05T07:44:43.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-mon[29507]: osdmap e2: 0 total, 0 up, 0 in 2024-02-05T07:44:43.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-mon[29507]: mgrmap e6: y(active, starting, since 0.00329867s) 2024-02-05T07:44:43.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:44:43.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T07:44:43.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T07:44:43.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T07:44:43.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T07:44:43.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-mon[29507]: Manager daemon y is now available 2024-02-05T07:44:43.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' 2024-02-05T07:44:43.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' 2024-02-05T07:44:43.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:44:43.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:42 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:44:43.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:43 smithi131 ceph-mon[29507]: Found migration_current of "None". Setting to last migration. 2024-02-05T07:44:43.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:43 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:44:43.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:43 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:44:43.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:43 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:44:43.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:43 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' 2024-02-05T07:44:43.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:43 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' 2024-02-05T07:44:43.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:43 smithi131 ceph-mon[29507]: mgrmap e7: y(active, since 1.00564s) 2024-02-05T07:44:44.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-02-05T07:44:44.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-05T07:44:44.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-05T07:44:44.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-02-05T07:44:44.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.775+0000 7f71b6ccd700 1 Processor -- start 2024-02-05T07:44:44.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.775+0000 7f71b6ccd700 1 -- start start 2024-02-05T07:44:44.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.775+0000 7f71b6ccd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b01068b0 0x7f71b0108ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:44.264 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.775+0000 7f71b6ccd700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b0078e70 con 0x7f71b01068b0 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.776+0000 7f71b4a69700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b01068b0 0x7f71b0108ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.776+0000 7f71b4a69700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b01068b0 0x7f71b0108ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58520/0 (socket says 172.21.15.131:58520) 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.776+0000 7f71b4a69700 1 -- 172.21.15.131:0/2597736604 learned_addr learned my addr 172.21.15.131:0/2597736604 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.776+0000 7f71b4a69700 1 -- 172.21.15.131:0/2597736604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b01091e0 con 0x7f71b01068b0 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.776+0000 7f71b4a69700 1 --2- 172.21.15.131:0/2597736604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b01068b0 0x7f71b0108ca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f71a40098a0 tx=0x7f71a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a14112b0e97bce5 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.776+0000 7f71af7fe700 1 -- 172.21.15.131:0/2597736604 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71a4004160 con 0x7f71b01068b0 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.776+0000 7f71af7fe700 1 -- 172.21.15.131:0/2597736604 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f71a40042c0 con 0x7f71b01068b0 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.776+0000 7f71af7fe700 1 -- 172.21.15.131:0/2597736604 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71a40119b0 con 0x7f71b01068b0 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.777+0000 7f71b6ccd700 1 -- 172.21.15.131:0/2597736604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b01068b0 msgr2=0x7f71b0108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.777+0000 7f71b6ccd700 1 --2- 172.21.15.131:0/2597736604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b01068b0 0x7f71b0108ca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f71a40098a0 tx=0x7f71a4009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.777+0000 7f71b6ccd700 1 -- 172.21.15.131:0/2597736604 shutdown_connections 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.777+0000 7f71b6ccd700 1 --2- 172.21.15.131:0/2597736604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b01068b0 0x7f71b0108ca0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.777+0000 7f71b6ccd700 1 -- 172.21.15.131:0/2597736604 >> 172.21.15.131:0/2597736604 conn(0x7f71b01002c0 msgr2=0x7f71b01026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.777+0000 7f71b6ccd700 1 -- 172.21.15.131:0/2597736604 shutdown_connections 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.777+0000 7f71b6ccd700 1 -- 172.21.15.131:0/2597736604 wait complete. 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.778+0000 7f71b6ccd700 1 Processor -- start 2024-02-05T07:44:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.778+0000 7f71b6ccd700 1 -- start start 2024-02-05T07:44:44.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.778+0000 7f71b6ccd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b01068b0 0x7f71b0106130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:44.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.778+0000 7f71b6ccd700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b0106670 con 0x7f71b01068b0 2024-02-05T07:44:44.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.778+0000 7f71b4a69700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b01068b0 0x7f71b0106130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:44.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.778+0000 7f71b4a69700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b01068b0 0x7f71b0106130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58522/0 (socket says 172.21.15.131:58522) 2024-02-05T07:44:44.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.778+0000 7f71b4a69700 1 -- 172.21.15.131:0/808389036 learned_addr learned my addr 172.21.15.131:0/808389036 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:44.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.778+0000 7f71b4a69700 1 -- 172.21.15.131:0/808389036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a4009550 con 0x7f71b01068b0 2024-02-05T07:44:44.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.778+0000 7f71b4a69700 1 --2- 172.21.15.131:0/808389036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b01068b0 0x7f71b0106130 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f71a40046c0 tx=0x7f71a4011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:44.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.779+0000 7f71adffb700 1 -- 172.21.15.131:0/808389036 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71a40039c0 con 0x7f71b01068b0 2024-02-05T07:44:44.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.779+0000 7f71adffb700 1 -- 172.21.15.131:0/808389036 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f71a4003b20 con 0x7f71b01068b0 2024-02-05T07:44:44.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.779+0000 7f71adffb700 1 -- 172.21.15.131:0/808389036 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71a401a7b0 con 0x7f71b01068b0 2024-02-05T07:44:44.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.779+0000 7f71b6ccd700 1 -- 172.21.15.131:0/808389036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b0104860 con 0x7f71b01068b0 2024-02-05T07:44:44.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.779+0000 7f71b6ccd700 1 -- 172.21.15.131:0/808389036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b0104d00 con 0x7f71b01068b0 2024-02-05T07:44:44.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.780+0000 7f71adffb700 1 -- 172.21.15.131:0/808389036 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) v1 ==== 45897+0+0 (secure 0 0 0) 0x7f71a4024b30 con 0x7f71b01068b0 2024-02-05T07:44:44.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.780+0000 7f71adffb700 1 --2- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/1416975616 conn(0x7f71a0038ea0 0x7f71a003b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:44.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.780+0000 7f71adffb700 1 -- 172.21.15.131:0/808389036 --> v2:172.21.15.131:6800/1416975616 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71a003ba30 con 0x7f71a0038ea0 2024-02-05T07:44:44.266 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.780+0000 7f71adffb700 1 -- 172.21.15.131:0/808389036 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f71a404cfc0 con 0x7f71b01068b0 2024-02-05T07:44:44.267 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.780+0000 7f71affff700 1 -- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/1416975616 conn(0x7f71a0038ea0 msgr2=0x7f71a003b360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1416975616 2024-02-05T07:44:44.267 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.780+0000 7f71affff700 1 --2- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/1416975616 conn(0x7f71a0038ea0 0x7f71a003b360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-05T07:44:44.267 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.981+0000 7f71affff700 1 -- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/1416975616 conn(0x7f71a0038ea0 msgr2=0x7f71a003b360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1416975616 2024-02-05T07:44:44.267 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:38.981+0000 7f71affff700 1 --2- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/1416975616 conn(0x7f71a0038ea0 0x7f71a003b360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-05T07:44:44.267 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:39.382+0000 7f71affff700 1 -- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/1416975616 conn(0x7f71a0038ea0 msgr2=0x7f71a003b360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1416975616 2024-02-05T07:44:44.267 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:39.382+0000 7f71affff700 1 --2- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/1416975616 conn(0x7f71a0038ea0 0x7f71a003b360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-05T07:44:44.267 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:40.183+0000 7f71affff700 1 -- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/1416975616 conn(0x7f71a0038ea0 msgr2=0x7f71a003b360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1416975616 2024-02-05T07:44:44.267 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:40.183+0000 7f71affff700 1 --2- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/1416975616 conn(0x7f71a0038ea0 0x7f71a003b360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-05T07:44:44.267 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:41.784+0000 7f71affff700 1 -- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/1416975616 conn(0x7f71a0038ea0 msgr2=0x7f71a003b360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/1416975616 2024-02-05T07:44:44.267 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:41.784+0000 7f71affff700 1 --2- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/1416975616 conn(0x7f71a0038ea0 0x7f71a003b360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-05T07:44:44.267 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:42.599+0000 7f71adffb700 1 -- 172.21.15.131:0/808389036 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mgrmap(e 6) v1 ==== 45699+0+0 (secure 0 0 0) 0x7f71a4024610 con 0x7f71b01068b0 2024-02-05T07:44:44.267 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:42.599+0000 7f71adffb700 1 -- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/1416975616 conn(0x7f71a0038ea0 msgr2=0x7f71a003b360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-05T07:44:44.267 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:42.599+0000 7f71adffb700 1 --2- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/1416975616 conn(0x7f71a0038ea0 0x7f71a003b360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:44.267 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.600+0000 7f71adffb700 1 -- 172.21.15.131:0/808389036 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 7) v1 ==== 45791+0+0 (secure 0 0 0) 0x7f71a402ace0 con 0x7f71b01068b0 2024-02-05T07:44:44.267 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.600+0000 7f71adffb700 1 --2- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/2947584178 conn(0x7f71a003c760 0x7f71a003eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:44.267 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.600+0000 7f71adffb700 1 -- 172.21.15.131:0/808389036 --> v2:172.21.15.131:6800/2947584178 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71a003ba30 con 0x7f71a003c760 2024-02-05T07:44:44.268 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.601+0000 7f71affff700 1 --2- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/2947584178 conn(0x7f71a003c760 0x7f71a003eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:44.268 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.601+0000 7f71affff700 1 --2- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/2947584178 conn(0x7f71a003c760 0x7f71a003eb50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f719c003a10 tx=0x7f719c0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:44.268 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.602+0000 7f71adffb700 1 -- 172.21.15.131:0/808389036 <== mgr.14120 v2:172.21.15.131:6800/2947584178 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f71a003ba30 con 0x7f71a003c760 2024-02-05T07:44:44.268 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.608+0000 7f71b6ccd700 1 -- 172.21.15.131:0/808389036 --> v2:172.21.15.131:6800/2947584178 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f71b0063350 con 0x7f71a003c760 2024-02-05T07:44:44.268 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.609+0000 7f71adffb700 1 -- 172.21.15.131:0/808389036 <== mgr.14120 v2:172.21.15.131:6800/2947584178 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f71b0063350 con 0x7f71a003c760 2024-02-05T07:44:44.268 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.609+0000 7f71b6ccd700 1 -- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/2947584178 conn(0x7f71a003c760 msgr2=0x7f71a003eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:44.268 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.609+0000 7f71b6ccd700 1 --2- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/2947584178 conn(0x7f71a003c760 0x7f71a003eb50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f719c003a10 tx=0x7f719c0092b0 comp rx=0 tx=0).stop 2024-02-05T07:44:44.268 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.609+0000 7f71b6ccd700 1 -- 172.21.15.131:0/808389036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b01068b0 msgr2=0x7f71b0106130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:44.268 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.609+0000 7f71b6ccd700 1 --2- 172.21.15.131:0/808389036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b01068b0 0x7f71b0106130 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f71a40046c0 tx=0x7f71a4011a50 comp rx=0 tx=0).stop 2024-02-05T07:44:44.268 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.609+0000 7f71b6ccd700 1 -- 172.21.15.131:0/808389036 shutdown_connections 2024-02-05T07:44:44.268 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.609+0000 7f71b6ccd700 1 --2- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/2947584178 conn(0x7f71a003c760 0x7f71a003eb50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:44.268 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.609+0000 7f71b6ccd700 1 --2- 172.21.15.131:0/808389036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71b01068b0 0x7f71b0106130 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:44.268 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.609+0000 7f71b6ccd700 1 --2- 172.21.15.131:0/808389036 >> v2:172.21.15.131:6800/1416975616 conn(0x7f71a0038ea0 0x7f71a003b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:44.268 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.609+0000 7f71b6ccd700 1 -- 172.21.15.131:0/808389036 >> 172.21.15.131:0/808389036 conn(0x7f71b01002c0 msgr2=0x7f71b0100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:44.268 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.609+0000 7f71b6ccd700 1 -- 172.21.15.131:0/808389036 shutdown_connections 2024-02-05T07:44:44.269 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:43.609+0000 7f71b6ccd700 1 -- 172.21.15.131:0/808389036 wait complete. 2024-02-05T07:44:44.269 INFO:teuthology.orchestra.run.smithi131.stdout:mgr epoch 5 is available 2024-02-05T07:44:44.269 INFO:teuthology.orchestra.run.smithi131.stdout:Setting orchestrator backend to cephadm... 2024-02-05T07:44:44.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:44 smithi131 ceph-mon[29507]: [05/Feb/2024:07:44:43] ENGINE Bus STARTING 2024-02-05T07:44:44.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:44 smithi131 ceph-mon[29507]: [05/Feb/2024:07:44:43] ENGINE Serving on https://172.21.15.131:7150 2024-02-05T07:44:44.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:44 smithi131 ceph-mon[29507]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-05T07:44:44.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:44 smithi131 ceph-mon[29507]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-05T07:44:44.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:44 smithi131 ceph-mon[29507]: [05/Feb/2024:07:44:43] ENGINE Serving on http://172.21.15.131:8765 2024-02-05T07:44:44.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:44 smithi131 ceph-mon[29507]: [05/Feb/2024:07:44:43] ENGINE Bus STARTED 2024-02-05T07:44:44.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:44 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:44:45.533 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.733+0000 7f1ebbbb5700 1 Processor -- start 2024-02-05T07:44:45.533 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.733+0000 7f1ebbbb5700 1 -- start start 2024-02-05T07:44:45.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.733+0000 7f1ebbbb5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1eb407b730 0x7f1eb4079b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:45.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.733+0000 7f1ebbbb5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb407a160 con 0x7f1eb407b730 2024-02-05T07:44:45.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.734+0000 7f1eb9951700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1eb407b730 0x7f1eb4079b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:45.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.734+0000 7f1eb9951700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1eb407b730 0x7f1eb4079b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58580/0 (socket says 172.21.15.131:58580) 2024-02-05T07:44:45.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.734+0000 7f1eb9951700 1 -- 172.21.15.131:0/1613391111 learned_addr learned my addr 172.21.15.131:0/1613391111 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:45.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.734+0000 7f1eb9951700 1 -- 172.21.15.131:0/1613391111 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb407a2a0 con 0x7f1eb407b730 2024-02-05T07:44:45.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.734+0000 7f1eb9951700 1 --2- 172.21.15.131:0/1613391111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1eb407b730 0x7f1eb4079b90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00098a0 tx=0x7f1eb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d581eef734dcf12 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:45.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.734+0000 7f1eb894f700 1 -- 172.21.15.131:0/1613391111 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1eb0004160 con 0x7f1eb407b730 2024-02-05T07:44:45.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.734+0000 7f1eb894f700 1 -- 172.21.15.131:0/1613391111 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f1eb00042c0 con 0x7f1eb407b730 2024-02-05T07:44:45.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.734+0000 7f1eb894f700 1 -- 172.21.15.131:0/1613391111 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1eb00119b0 con 0x7f1eb407b730 2024-02-05T07:44:45.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.735+0000 7f1ebbbb5700 1 -- 172.21.15.131:0/1613391111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1eb407b730 msgr2=0x7f1eb4079b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:45.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.735+0000 7f1ebbbb5700 1 --2- 172.21.15.131:0/1613391111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1eb407b730 0x7f1eb4079b90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00098a0 tx=0x7f1eb0009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:45.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.735+0000 7f1ebbbb5700 1 -- 172.21.15.131:0/1613391111 shutdown_connections 2024-02-05T07:44:45.534 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.735+0000 7f1ebbbb5700 1 --2- 172.21.15.131:0/1613391111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1eb407b730 0x7f1eb4079b90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.735+0000 7f1ebbbb5700 1 -- 172.21.15.131:0/1613391111 >> 172.21.15.131:0/1613391111 conn(0x7f1eb41013b0 msgr2=0x7f1eb4103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.735+0000 7f1ebbbb5700 1 -- 172.21.15.131:0/1613391111 shutdown_connections 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.735+0000 7f1ebbbb5700 1 -- 172.21.15.131:0/1613391111 wait complete. 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.735+0000 7f1ebbbb5700 1 Processor -- start 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.735+0000 7f1ebbbb5700 1 -- start start 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.735+0000 7f1ebbbb5700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1eb407b730 0x7f1eb419bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.735+0000 7f1ebbbb5700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb419c220 con 0x7f1eb407b730 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.735+0000 7f1eb9951700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1eb407b730 0x7f1eb419bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.736+0000 7f1eb9951700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1eb407b730 0x7f1eb419bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58582/0 (socket says 172.21.15.131:58582) 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.736+0000 7f1eb9951700 1 -- 172.21.15.131:0/1594607482 learned_addr learned my addr 172.21.15.131:0/1594607482 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.736+0000 7f1eb9951700 1 -- 172.21.15.131:0/1594607482 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb0009550 con 0x7f1eb407b730 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.736+0000 7f1eb9951700 1 --2- 172.21.15.131:0/1594607482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1eb407b730 0x7f1eb419bce0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00046c0 tx=0x7f1eb0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.736+0000 7f1eaaffd700 1 -- 172.21.15.131:0/1594607482 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1eb00038b0 con 0x7f1eb407b730 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.736+0000 7f1ebbbb5700 1 -- 172.21.15.131:0/1594607482 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb419c420 con 0x7f1eb407b730 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.736+0000 7f1eaaffd700 1 -- 172.21.15.131:0/1594607482 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f1eb0003a10 con 0x7f1eb407b730 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.736+0000 7f1eaaffd700 1 -- 172.21.15.131:0/1594607482 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1eb0019610 con 0x7f1eb407b730 2024-02-05T07:44:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.736+0000 7f1ebbbb5700 1 -- 172.21.15.131:0/1594607482 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb419c8c0 con 0x7f1eb407b730 2024-02-05T07:44:45.536 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.737+0000 7f1eaaffd700 1 -- 172.21.15.131:0/1594607482 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7f1eb001a430 con 0x7f1eb407b730 2024-02-05T07:44:45.536 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.737+0000 7f1ebbbb5700 1 -- 172.21.15.131:0/1594607482 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb410c150 con 0x7f1eb407b730 2024-02-05T07:44:45.536 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.737+0000 7f1eaaffd700 1 --2- 172.21.15.131:0/1594607482 >> v2:172.21.15.131:6800/2947584178 conn(0x7f1ea0038e50 0x7f1ea003b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:45.536 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.737+0000 7f1eaaffd700 1 -- 172.21.15.131:0/1594607482 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f1eb004cf60 con 0x7f1eb407b730 2024-02-05T07:44:45.536 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.737+0000 7f1eb9150700 1 --2- 172.21.15.131:0/1594607482 >> v2:172.21.15.131:6800/2947584178 conn(0x7f1ea0038e50 0x7f1ea003b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:45.536 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.737+0000 7f1eb9150700 1 --2- 172.21.15.131:0/1594607482 >> v2:172.21.15.131:6800/2947584178 conn(0x7f1ea0038e50 0x7f1ea003b310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009940 tx=0x7f1ea4006e30 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:45.536 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.742+0000 7f1eaaffd700 1 -- 172.21.15.131:0/1594607482 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1eb0023390 con 0x7f1eb407b730 2024-02-05T07:44:45.536 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.950+0000 7f1ebbbb5700 1 -- 172.21.15.131:0/1594607482 --> v2:172.21.15.131:6800/2947584178 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f1eb41026d0 con 0x7f1ea0038e50 2024-02-05T07:44:45.536 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.955+0000 7f1eaaffd700 1 -- 172.21.15.131:0/1594607482 <== mgr.14120 v2:172.21.15.131:6800/2947584178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1eb41026d0 con 0x7f1ea0038e50 2024-02-05T07:44:45.536 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.958+0000 7f1ebbbb5700 1 -- 172.21.15.131:0/1594607482 >> v2:172.21.15.131:6800/2947584178 conn(0x7f1ea0038e50 msgr2=0x7f1ea003b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:45.536 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.958+0000 7f1ebbbb5700 1 --2- 172.21.15.131:0/1594607482 >> v2:172.21.15.131:6800/2947584178 conn(0x7f1ea0038e50 0x7f1ea003b310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009940 tx=0x7f1ea4006e30 comp rx=0 tx=0).stop 2024-02-05T07:44:45.536 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.958+0000 7f1ebbbb5700 1 -- 172.21.15.131:0/1594607482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1eb407b730 msgr2=0x7f1eb419bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:45.536 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.958+0000 7f1ebbbb5700 1 --2- 172.21.15.131:0/1594607482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1eb407b730 0x7f1eb419bce0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00046c0 tx=0x7f1eb0011a50 comp rx=0 tx=0).stop 2024-02-05T07:44:45.536 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.958+0000 7f1ebbbb5700 1 -- 172.21.15.131:0/1594607482 shutdown_connections 2024-02-05T07:44:45.536 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.958+0000 7f1ebbbb5700 1 --2- 172.21.15.131:0/1594607482 >> v2:172.21.15.131:6800/2947584178 conn(0x7f1ea0038e50 0x7f1ea003b310 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:45.536 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.958+0000 7f1ebbbb5700 1 --2- 172.21.15.131:0/1594607482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1eb407b730 0x7f1eb419bce0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:45.537 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.958+0000 7f1ebbbb5700 1 -- 172.21.15.131:0/1594607482 >> 172.21.15.131:0/1594607482 conn(0x7f1eb41013b0 msgr2=0x7f1eb4101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:45.537 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.958+0000 7f1ebbbb5700 1 -- 172.21.15.131:0/1594607482 shutdown_connections 2024-02-05T07:44:45.537 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:44.958+0000 7f1ebbbb5700 1 -- 172.21.15.131:0/1594607482 wait complete. 2024-02-05T07:44:45.781 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:45 smithi131 ceph-mon[29507]: mgrmap e8: y(active, since 2s) 2024-02-05T07:44:45.781 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:45 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' 2024-02-05T07:44:45.781 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:45 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:44:46.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-05T07:44:46.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.987+0000 7fc066c54700 1 Processor -- start 2024-02-05T07:44:46.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.987+0000 7fc066c54700 1 -- start start 2024-02-05T07:44:46.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.987+0000 7fc066c54700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc06007b730 0x7fc060079b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:46.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.987+0000 7fc066c54700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06007a160 con 0x7fc06007b730 2024-02-05T07:44:46.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.987+0000 7fc0649f0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc06007b730 0x7fc060079b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:46.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.987+0000 7fc0649f0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc06007b730 0x7fc060079b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58598/0 (socket says 172.21.15.131:58598) 2024-02-05T07:44:46.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.987+0000 7fc0649f0700 1 -- 172.21.15.131:0/3129653242 learned_addr learned my addr 172.21.15.131:0/3129653242 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:46.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.988+0000 7fc0649f0700 1 -- 172.21.15.131:0/3129653242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06007a2a0 con 0x7fc06007b730 2024-02-05T07:44:46.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.988+0000 7fc0649f0700 1 --2- 172.21.15.131:0/3129653242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc06007b730 0x7fc060079b90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc0540098a0 tx=0x7fc054009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ba19c4c2fe098b server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:46.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.988+0000 7fc05f7fe700 1 -- 172.21.15.131:0/3129653242 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc054004160 con 0x7fc06007b730 2024-02-05T07:44:46.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.988+0000 7fc05f7fe700 1 -- 172.21.15.131:0/3129653242 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fc0540042c0 con 0x7fc06007b730 2024-02-05T07:44:46.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.988+0000 7fc05f7fe700 1 -- 172.21.15.131:0/3129653242 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0540119b0 con 0x7fc06007b730 2024-02-05T07:44:46.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.988+0000 7fc066c54700 1 -- 172.21.15.131:0/3129653242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc06007b730 msgr2=0x7fc060079b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.988+0000 7fc066c54700 1 --2- 172.21.15.131:0/3129653242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc06007b730 0x7fc060079b90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc0540098a0 tx=0x7fc054009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.989+0000 7fc066c54700 1 -- 172.21.15.131:0/3129653242 shutdown_connections 2024-02-05T07:44:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.989+0000 7fc066c54700 1 --2- 172.21.15.131:0/3129653242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc06007b730 0x7fc060079b90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.989+0000 7fc066c54700 1 -- 172.21.15.131:0/3129653242 >> 172.21.15.131:0/3129653242 conn(0x7fc0601014d0 msgr2=0x7fc0601038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.989+0000 7fc066c54700 1 -- 172.21.15.131:0/3129653242 shutdown_connections 2024-02-05T07:44:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.989+0000 7fc066c54700 1 -- 172.21.15.131:0/3129653242 wait complete. 2024-02-05T07:44:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.989+0000 7fc066c54700 1 Processor -- start 2024-02-05T07:44:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.989+0000 7fc066c54700 1 -- start start 2024-02-05T07:44:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.989+0000 7fc066c54700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc06007b730 0x7fc06019bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.989+0000 7fc066c54700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06019c230 con 0x7fc06007b730 2024-02-05T07:44:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.989+0000 7fc0649f0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc06007b730 0x7fc06019bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.989+0000 7fc0649f0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc06007b730 0x7fc06019bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58610/0 (socket says 172.21.15.131:58610) 2024-02-05T07:44:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.989+0000 7fc0649f0700 1 -- 172.21.15.131:0/2340278033 learned_addr learned my addr 172.21.15.131:0/2340278033 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.990+0000 7fc0649f0700 1 -- 172.21.15.131:0/2340278033 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc054009550 con 0x7fc06007b730 2024-02-05T07:44:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.990+0000 7fc0649f0700 1 --2- 172.21.15.131:0/2340278033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc06007b730 0x7fc06019bcf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc054006af0 tx=0x7fc054011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.990+0000 7fc05dffb700 1 -- 172.21.15.131:0/2340278033 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0540039e0 con 0x7fc06007b730 2024-02-05T07:44:46.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.990+0000 7fc066c54700 1 -- 172.21.15.131:0/2340278033 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06019c430 con 0x7fc06007b730 2024-02-05T07:44:46.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.990+0000 7fc05dffb700 1 -- 172.21.15.131:0/2340278033 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fc054003b40 con 0x7fc06007b730 2024-02-05T07:44:46.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.990+0000 7fc05dffb700 1 -- 172.21.15.131:0/2340278033 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc054019700 con 0x7fc06007b730 2024-02-05T07:44:46.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.990+0000 7fc066c54700 1 -- 172.21.15.131:0/2340278033 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc06019c8d0 con 0x7fc06007b730 2024-02-05T07:44:46.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.991+0000 7fc05dffb700 1 -- 172.21.15.131:0/2340278033 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7fc054019930 con 0x7fc06007b730 2024-02-05T07:44:46.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.991+0000 7fc066c54700 1 -- 172.21.15.131:0/2340278033 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06010c1b0 con 0x7fc06007b730 2024-02-05T07:44:46.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.991+0000 7fc05dffb700 1 --2- 172.21.15.131:0/2340278033 >> v2:172.21.15.131:6800/2947584178 conn(0x7fc050038ea0 0x7fc05003b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:46.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.991+0000 7fc05dffb700 1 -- 172.21.15.131:0/2340278033 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7fc05404cef0 con 0x7fc06007b730 2024-02-05T07:44:46.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.991+0000 7fc05ffff700 1 --2- 172.21.15.131:0/2340278033 >> v2:172.21.15.131:6800/2947584178 conn(0x7fc050038ea0 0x7fc05003b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:46.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.992+0000 7fc05ffff700 1 --2- 172.21.15.131:0/2340278033 >> v2:172.21.15.131:6800/2947584178 conn(0x7fc050038ea0 0x7fc05003b360 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc04c006fd0 tx=0x7fc04c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:46.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:45.996+0000 7fc05dffb700 1 -- 172.21.15.131:0/2340278033 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc05402c3f0 con 0x7fc06007b730 2024-02-05T07:44:46.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:46.191+0000 7fc066c54700 1 -- 172.21.15.131:0/2340278033 --> v2:172.21.15.131:6800/2947584178 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fc060102740 con 0x7fc050038ea0 2024-02-05T07:44:46.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:46.192+0000 7fc05dffb700 1 -- 172.21.15.131:0/2340278033 <== mgr.14120 v2:172.21.15.131:6800/2947584178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fc060102740 con 0x7fc050038ea0 2024-02-05T07:44:46.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:46.194+0000 7fc066c54700 1 -- 172.21.15.131:0/2340278033 >> v2:172.21.15.131:6800/2947584178 conn(0x7fc050038ea0 msgr2=0x7fc05003b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:46.825 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:46.194+0000 7fc066c54700 1 --2- 172.21.15.131:0/2340278033 >> v2:172.21.15.131:6800/2947584178 conn(0x7fc050038ea0 0x7fc05003b360 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc04c006fd0 tx=0x7fc04c006e40 comp rx=0 tx=0).stop 2024-02-05T07:44:46.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:46.194+0000 7fc066c54700 1 -- 172.21.15.131:0/2340278033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc06007b730 msgr2=0x7fc06019bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:46.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:46.194+0000 7fc066c54700 1 --2- 172.21.15.131:0/2340278033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc06007b730 0x7fc06019bcf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc054006af0 tx=0x7fc054011a50 comp rx=0 tx=0).stop 2024-02-05T07:44:46.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:46.194+0000 7fc066c54700 1 -- 172.21.15.131:0/2340278033 shutdown_connections 2024-02-05T07:44:46.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:46.194+0000 7fc066c54700 1 --2- 172.21.15.131:0/2340278033 >> v2:172.21.15.131:6800/2947584178 conn(0x7fc050038ea0 0x7fc05003b360 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:46.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:46.194+0000 7fc066c54700 1 --2- 172.21.15.131:0/2340278033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc06007b730 0x7fc06019bcf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:46.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:46.195+0000 7fc066c54700 1 -- 172.21.15.131:0/2340278033 >> 172.21.15.131:0/2340278033 conn(0x7fc0601014d0 msgr2=0x7fc060102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:46.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:46.195+0000 7fc066c54700 1 -- 172.21.15.131:0/2340278033 shutdown_connections 2024-02-05T07:44:46.826 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:46.195+0000 7fc066c54700 1 -- 172.21.15.131:0/2340278033 wait complete. 2024-02-05T07:44:46.826 INFO:teuthology.orchestra.run.smithi131.stdout:Generating ssh key... 2024-02-05T07:44:46.942 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:46 smithi131 ceph-mon[29507]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:44:48.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-mon[29507]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:44:48.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: Generating public/private rsa key pair. 2024-02-05T07:44:48.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: Your identification has been saved in /tmp/tmpbt5v2rfm/key. 2024-02-05T07:44:48.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: Your public key has been saved in /tmp/tmpbt5v2rfm/key.pub. 2024-02-05T07:44:48.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: The key fingerprint is: 2024-02-05T07:44:48.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: SHA256:NdCZmatR79Ohvdup2AG9sU4c2g7zjD6FUaobPG3jHXw ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:44:48.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: The key's randomart image is: 2024-02-05T07:44:48.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: +---[RSA 3072]----+ 2024-02-05T07:44:48.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: | .. = | 2024-02-05T07:44:48.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: | .B . | 2024-02-05T07:44:48.019 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: | .ooo | 2024-02-05T07:44:48.019 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: | ...+o . | 2024-02-05T07:44:48.019 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: | Sooo+B . | 2024-02-05T07:44:48.019 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: | .= =O+BE | 2024-02-05T07:44:48.019 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: | *++Oo. | 2024-02-05T07:44:48.019 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: | . o%.o..| 2024-02-05T07:44:48.019 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: | .+.Boo.| 2024-02-05T07:44:48.019 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:44:47 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: +----[SHA256]-----+ 2024-02-05T07:44:48.454 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.284+0000 7f5b4600a700 1 Processor -- start 2024-02-05T07:44:48.455 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.284+0000 7f5b4600a700 1 -- start start 2024-02-05T07:44:48.455 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.284+0000 7f5b4600a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b401062b0 0x7f5b401066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:48.455 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.284+0000 7f5b4600a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b40106ca0 con 0x7f5b401062b0 2024-02-05T07:44:48.455 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.285+0000 7f5b3f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b401062b0 0x7f5b401066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:48.455 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.285+0000 7f5b3f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b401062b0 0x7f5b401066d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58618/0 (socket says 172.21.15.131:58618) 2024-02-05T07:44:48.455 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.285+0000 7f5b3f7fe700 1 -- 172.21.15.131:0/242696110 learned_addr learned my addr 172.21.15.131:0/242696110 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:48.455 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.285+0000 7f5b3f7fe700 1 -- 172.21.15.131:0/242696110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b40104f10 con 0x7f5b401062b0 2024-02-05T07:44:48.455 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.285+0000 7f5b3f7fe700 1 --2- 172.21.15.131:0/242696110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b401062b0 0x7f5b401066d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5b300098a0 tx=0x7f5b30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72bf62511d98f server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:48.455 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.285+0000 7f5b3e7fc700 1 -- 172.21.15.131:0/242696110 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b30004160 con 0x7f5b401062b0 2024-02-05T07:44:48.455 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.285+0000 7f5b3e7fc700 1 -- 172.21.15.131:0/242696110 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f5b300042c0 con 0x7f5b401062b0 2024-02-05T07:44:48.455 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.285+0000 7f5b3e7fc700 1 -- 172.21.15.131:0/242696110 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b300119b0 con 0x7f5b401062b0 2024-02-05T07:44:48.455 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.286+0000 7f5b4600a700 1 -- 172.21.15.131:0/242696110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b401062b0 msgr2=0x7f5b401066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:48.455 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.286+0000 7f5b4600a700 1 --2- 172.21.15.131:0/242696110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b401062b0 0x7f5b401066d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5b300098a0 tx=0x7f5b30009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:48.455 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.286+0000 7f5b4600a700 1 -- 172.21.15.131:0/242696110 shutdown_connections 2024-02-05T07:44:48.455 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.286+0000 7f5b4600a700 1 --2- 172.21.15.131:0/242696110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b401062b0 0x7f5b401066d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:48.456 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.286+0000 7f5b4600a700 1 -- 172.21.15.131:0/242696110 >> 172.21.15.131:0/242696110 conn(0x7f5b400ffa40 msgr2=0x7f5b40101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:48.456 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.286+0000 7f5b4600a700 1 -- 172.21.15.131:0/242696110 shutdown_connections 2024-02-05T07:44:48.456 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.286+0000 7f5b4600a700 1 -- 172.21.15.131:0/242696110 wait complete. 2024-02-05T07:44:48.456 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.287+0000 7f5b4600a700 1 Processor -- start 2024-02-05T07:44:48.456 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.287+0000 7f5b4600a700 1 -- start start 2024-02-05T07:44:48.456 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.287+0000 7f5b4600a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b401062b0 0x7f5b40199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:48.456 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.287+0000 7f5b4600a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4019a020 con 0x7f5b401062b0 2024-02-05T07:44:48.456 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.287+0000 7f5b3f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b401062b0 0x7f5b40199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:48.456 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.287+0000 7f5b3f7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b401062b0 0x7f5b40199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58632/0 (socket says 172.21.15.131:58632) 2024-02-05T07:44:48.456 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.287+0000 7f5b3f7fe700 1 -- 172.21.15.131:0/3942732754 learned_addr learned my addr 172.21.15.131:0/3942732754 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:48.456 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.288+0000 7f5b3f7fe700 1 -- 172.21.15.131:0/3942732754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b30009550 con 0x7f5b401062b0 2024-02-05T07:44:48.456 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.288+0000 7f5b3f7fe700 1 --2- 172.21.15.131:0/3942732754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b401062b0 0x7f5b40199ae0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5b300046c0 tx=0x7f5b30011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:48.456 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.288+0000 7f5b3cff9700 1 -- 172.21.15.131:0/3942732754 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b300039c0 con 0x7f5b401062b0 2024-02-05T07:44:48.456 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.288+0000 7f5b4600a700 1 -- 172.21.15.131:0/3942732754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4019a220 con 0x7f5b401062b0 2024-02-05T07:44:48.456 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.288+0000 7f5b3cff9700 1 -- 172.21.15.131:0/3942732754 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f5b30003b20 con 0x7f5b401062b0 2024-02-05T07:44:48.457 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.288+0000 7f5b3cff9700 1 -- 172.21.15.131:0/3942732754 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b3001a7b0 con 0x7f5b401062b0 2024-02-05T07:44:48.457 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.288+0000 7f5b4600a700 1 -- 172.21.15.131:0/3942732754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4019a6c0 con 0x7f5b401062b0 2024-02-05T07:44:48.457 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.289+0000 7f5b3cff9700 1 -- 172.21.15.131:0/3942732754 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7f5b3001aa20 con 0x7f5b401062b0 2024-02-05T07:44:48.457 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.289+0000 7f5b3cff9700 1 --2- 172.21.15.131:0/3942732754 >> v2:172.21.15.131:6800/2947584178 conn(0x7f5b28038e50 0x7f5b2803b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:48.457 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.289+0000 7f5b3cff9700 1 -- 172.21.15.131:0/3942732754 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f5b3004cd20 con 0x7f5b401062b0 2024-02-05T07:44:48.457 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.289+0000 7f5b3effd700 1 --2- 172.21.15.131:0/3942732754 >> v2:172.21.15.131:6800/2947584178 conn(0x7f5b28038e50 0x7f5b2803b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:48.457 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.289+0000 7f5b4600a700 1 -- 172.21.15.131:0/3942732754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b40193bb0 con 0x7f5b401062b0 2024-02-05T07:44:48.457 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.289+0000 7f5b3effd700 1 --2- 172.21.15.131:0/3942732754 >> v2:172.21.15.131:6800/2947584178 conn(0x7f5b28038e50 0x7f5b2803b310 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5b34006fd0 tx=0x7f5b34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:48.457 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.294+0000 7f5b3cff9700 1 -- 172.21.15.131:0/3942732754 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b300237a0 con 0x7f5b401062b0 2024-02-05T07:44:48.457 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.512+0000 7f5b4600a700 1 -- 172.21.15.131:0/3942732754 --> v2:172.21.15.131:6800/2947584178 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b40062160 con 0x7f5b28038e50 2024-02-05T07:44:48.457 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.828+0000 7f5b3cff9700 1 -- 172.21.15.131:0/3942732754 <== mgr.14120 v2:172.21.15.131:6800/2947584178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5b40062160 con 0x7f5b28038e50 2024-02-05T07:44:48.457 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.830+0000 7f5b4600a700 1 -- 172.21.15.131:0/3942732754 >> v2:172.21.15.131:6800/2947584178 conn(0x7f5b28038e50 msgr2=0x7f5b2803b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:48.457 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.830+0000 7f5b4600a700 1 --2- 172.21.15.131:0/3942732754 >> v2:172.21.15.131:6800/2947584178 conn(0x7f5b28038e50 0x7f5b2803b310 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5b34006fd0 tx=0x7f5b34006e40 comp rx=0 tx=0).stop 2024-02-05T07:44:48.457 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.830+0000 7f5b4600a700 1 -- 172.21.15.131:0/3942732754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b401062b0 msgr2=0x7f5b40199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:48.457 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.830+0000 7f5b4600a700 1 --2- 172.21.15.131:0/3942732754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b401062b0 0x7f5b40199ae0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5b300046c0 tx=0x7f5b30011a50 comp rx=0 tx=0).stop 2024-02-05T07:44:48.457 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.830+0000 7f5b4600a700 1 -- 172.21.15.131:0/3942732754 shutdown_connections 2024-02-05T07:44:48.458 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.830+0000 7f5b4600a700 1 --2- 172.21.15.131:0/3942732754 >> v2:172.21.15.131:6800/2947584178 conn(0x7f5b28038e50 0x7f5b2803b310 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:48.458 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.830+0000 7f5b4600a700 1 --2- 172.21.15.131:0/3942732754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b401062b0 0x7f5b40199ae0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:48.458 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.830+0000 7f5b4600a700 1 -- 172.21.15.131:0/3942732754 >> 172.21.15.131:0/3942732754 conn(0x7f5b400ffa40 msgr2=0x7f5b40100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:48.458 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.830+0000 7f5b4600a700 1 -- 172.21.15.131:0/3942732754 shutdown_connections 2024-02-05T07:44:48.458 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:47.830+0000 7f5b4600a700 1 -- 172.21.15.131:0/3942732754 wait complete. 2024-02-05T07:44:49.120 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:48 smithi131 ceph-mon[29507]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:44:49.120 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:48 smithi131 ceph-mon[29507]: Generating ssh key... 2024-02-05T07:44:49.120 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:48 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' 2024-02-05T07:44:49.120 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:48 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' 2024-02-05T07:44:49.659 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6kv1YeFWAV37cFlbLKm3kh5Y1F8XrvmDb89VWrFCVhuUA43fHh+iQTpx5Xt7kgD6K1jx5p6ja8gGnh29ul4O3hsliYWNL+wehjYy55GafZRdQmYHcCCcyg6ViN/rkC05v8K1HpGisqU1A6NhjRrC6yEPWVmASx+ejM7KBaADk+aa2Sl7nVVX2olZXveaNzMD1rnVIdE1eXySPvcvwQc1unYotafdm70wXJAWgoLFWQ2FKsDiXv2A9o/GR0JSzSBqSBGjuhYj0qPBLNb+86PFVnD05jhTmTrpK+BghKS0N19jCBjmiiLPC3LIjqgURBBHb/F29Pl8rOFm0KjoGUlx2Y7mGQjRweI6XSwZghGmKJZ6WBRGZwMffP4x6GZNvNaEnE8Tjck1xqKq3xvNLJ/28kuEPEA91+3Q5Cj6rYnsos8Bt2i8RO5kNW6oHLHMcAS/WhLOdSfSAmxhA+f84OWrxno+M4xBK0PX92ZQpbS6nRreoZaS2DmRKPDIWzS1RBLE= ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:44:49.659 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.916+0000 7ff46438c700 1 Processor -- start 2024-02-05T07:44:49.659 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.916+0000 7ff46438c700 1 -- start start 2024-02-05T07:44:49.659 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.917+0000 7ff46438c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c106960 0x7ff45c108d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:49.659 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.917+0000 7ff46438c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c079700 con 0x7ff45c106960 2024-02-05T07:44:49.659 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.917+0000 7ff462128700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c106960 0x7ff45c108d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:49.659 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.917+0000 7ff462128700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c106960 0x7ff45c108d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54866/0 (socket says 172.21.15.131:54866) 2024-02-05T07:44:49.659 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.917+0000 7ff462128700 1 -- 172.21.15.131:0/989145320 learned_addr learned my addr 172.21.15.131:0/989145320 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:49.659 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.917+0000 7ff462128700 1 -- 172.21.15.131:0/989145320 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c109290 con 0x7ff45c106960 2024-02-05T07:44:49.659 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.917+0000 7ff462128700 1 --2- 172.21.15.131:0/989145320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c106960 0x7ff45c108d50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0098a0 tx=0x7ff44c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715f14e9ed31862d server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:49.659 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.917+0000 7ff461126700 1 -- 172.21.15.131:0/989145320 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff44c004160 con 0x7ff45c106960 2024-02-05T07:44:49.659 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.917+0000 7ff461126700 1 -- 172.21.15.131:0/989145320 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7ff44c0042c0 con 0x7ff45c106960 2024-02-05T07:44:49.660 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.917+0000 7ff461126700 1 -- 172.21.15.131:0/989145320 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff44c0119b0 con 0x7ff45c106960 2024-02-05T07:44:49.660 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.918+0000 7ff46438c700 1 -- 172.21.15.131:0/989145320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c106960 msgr2=0x7ff45c108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:49.660 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.918+0000 7ff46438c700 1 --2- 172.21.15.131:0/989145320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c106960 0x7ff45c108d50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0098a0 tx=0x7ff44c009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:49.660 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.918+0000 7ff46438c700 1 -- 172.21.15.131:0/989145320 shutdown_connections 2024-02-05T07:44:49.660 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.918+0000 7ff46438c700 1 --2- 172.21.15.131:0/989145320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c106960 0x7ff45c108d50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:49.660 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.918+0000 7ff46438c700 1 -- 172.21.15.131:0/989145320 >> 172.21.15.131:0/989145320 conn(0x7ff45c1003b0 msgr2=0x7ff45c1027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:49.660 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.918+0000 7ff46438c700 1 -- 172.21.15.131:0/989145320 shutdown_connections 2024-02-05T07:44:49.660 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.918+0000 7ff46438c700 1 -- 172.21.15.131:0/989145320 wait complete. 2024-02-05T07:44:49.660 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.919+0000 7ff46438c700 1 Processor -- start 2024-02-05T07:44:49.660 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.919+0000 7ff46438c700 1 -- start start 2024-02-05T07:44:49.660 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.919+0000 7ff46438c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c106960 0x7ff45c1061e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:49.660 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.919+0000 7ff46438c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c106720 con 0x7ff45c106960 2024-02-05T07:44:49.660 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.919+0000 7ff462128700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c106960 0x7ff45c1061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:49.660 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.919+0000 7ff462128700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c106960 0x7ff45c1061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54882/0 (socket says 172.21.15.131:54882) 2024-02-05T07:44:49.660 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.919+0000 7ff462128700 1 -- 172.21.15.131:0/4090206234 learned_addr learned my addr 172.21.15.131:0/4090206234 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:49.661 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.919+0000 7ff462128700 1 -- 172.21.15.131:0/4090206234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c009550 con 0x7ff45c106960 2024-02-05T07:44:49.661 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.919+0000 7ff462128700 1 --2- 172.21.15.131:0/4090206234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c106960 0x7ff45c1061e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff44c011670 tx=0x7ff44c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:49.661 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.919+0000 7ff4537fe700 1 -- 172.21.15.131:0/4090206234 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff44c003af0 con 0x7ff45c106960 2024-02-05T07:44:49.661 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.919+0000 7ff4537fe700 1 -- 172.21.15.131:0/4090206234 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7ff44c01a8c0 con 0x7ff45c106960 2024-02-05T07:44:49.661 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.919+0000 7ff46438c700 1 -- 172.21.15.131:0/4090206234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45c104910 con 0x7ff45c106960 2024-02-05T07:44:49.661 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.919+0000 7ff46438c700 1 -- 172.21.15.131:0/4090206234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45c104db0 con 0x7ff45c106960 2024-02-05T07:44:49.661 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.920+0000 7ff4537fe700 1 -- 172.21.15.131:0/4090206234 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff44c019730 con 0x7ff45c106960 2024-02-05T07:44:49.661 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.920+0000 7ff4537fe700 1 -- 172.21.15.131:0/4090206234 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7ff44c019a70 con 0x7ff45c106960 2024-02-05T07:44:49.661 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.921+0000 7ff46438c700 1 -- 172.21.15.131:0/4090206234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff440005150 con 0x7ff45c106960 2024-02-05T07:44:49.661 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.921+0000 7ff4537fe700 1 --2- 172.21.15.131:0/4090206234 >> v2:172.21.15.131:6800/2947584178 conn(0x7ff448038ea0 0x7ff44803b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:49.661 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.921+0000 7ff4537fe700 1 -- 172.21.15.131:0/4090206234 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7ff44c04d100 con 0x7ff45c106960 2024-02-05T07:44:49.661 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.921+0000 7ff461927700 1 --2- 172.21.15.131:0/4090206234 >> v2:172.21.15.131:6800/2947584178 conn(0x7ff448038ea0 0x7ff44803b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:49.661 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.921+0000 7ff461927700 1 --2- 172.21.15.131:0/4090206234 >> v2:172.21.15.131:6800/2947584178 conn(0x7ff448038ea0 0x7ff44803b360 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff458006fd0 tx=0x7ff458006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:49.661 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:48.926+0000 7ff4537fe700 1 -- 172.21.15.131:0/4090206234 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff44c018c20 con 0x7ff45c106960 2024-02-05T07:44:49.661 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:49.118+0000 7ff46438c700 1 -- 172.21.15.131:0/4090206234 --> v2:172.21.15.131:6800/2947584178 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7ff440000bf0 con 0x7ff448038ea0 2024-02-05T07:44:49.662 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:49.119+0000 7ff4537fe700 1 -- 172.21.15.131:0/4090206234 <== mgr.14120 v2:172.21.15.131:6800/2947584178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7ff440000bf0 con 0x7ff448038ea0 2024-02-05T07:44:49.662 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:49.121+0000 7ff46438c700 1 -- 172.21.15.131:0/4090206234 >> v2:172.21.15.131:6800/2947584178 conn(0x7ff448038ea0 msgr2=0x7ff44803b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:49.662 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:49.121+0000 7ff46438c700 1 --2- 172.21.15.131:0/4090206234 >> v2:172.21.15.131:6800/2947584178 conn(0x7ff448038ea0 0x7ff44803b360 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff458006fd0 tx=0x7ff458006e40 comp rx=0 tx=0).stop 2024-02-05T07:44:49.662 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:49.121+0000 7ff46438c700 1 -- 172.21.15.131:0/4090206234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c106960 msgr2=0x7ff45c1061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:49.662 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:49.121+0000 7ff46438c700 1 --2- 172.21.15.131:0/4090206234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c106960 0x7ff45c1061e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff44c011670 tx=0x7ff44c011a50 comp rx=0 tx=0).stop 2024-02-05T07:44:49.662 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:49.121+0000 7ff46438c700 1 -- 172.21.15.131:0/4090206234 shutdown_connections 2024-02-05T07:44:49.662 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:49.121+0000 7ff46438c700 1 --2- 172.21.15.131:0/4090206234 >> v2:172.21.15.131:6800/2947584178 conn(0x7ff448038ea0 0x7ff44803b360 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:49.662 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:49.121+0000 7ff46438c700 1 --2- 172.21.15.131:0/4090206234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c106960 0x7ff45c1061e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:49.662 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:49.121+0000 7ff46438c700 1 -- 172.21.15.131:0/4090206234 >> 172.21.15.131:0/4090206234 conn(0x7ff45c1003b0 msgr2=0x7ff45c100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:49.662 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:49.121+0000 7ff46438c700 1 -- 172.21.15.131:0/4090206234 shutdown_connections 2024-02-05T07:44:49.662 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:49.121+0000 7ff46438c700 1 -- 172.21.15.131:0/4090206234 wait complete. 2024-02-05T07:44:49.662 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-05T07:44:49.662 INFO:teuthology.orchestra.run.smithi131.stdout:Adding key to root@localhost authorized_keys... 2024-02-05T07:44:49.662 INFO:teuthology.orchestra.run.smithi131.stdout:Adding host smithi131... 2024-02-05T07:44:50.097 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:49 smithi131 ceph-mon[29507]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:44:51.165 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:50 smithi131 ceph-mon[29507]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi131", "addr": "172.21.15.131", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:44:52.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:51 smithi131 ceph-mon[29507]: Deploying cephadm binary to smithi131 2024-02-05T07:44:55.227 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Added host 'smithi131' with addr '172.21.15.131' 2024-02-05T07:44:55.227 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.107+0000 7f382d534700 1 Processor -- start 2024-02-05T07:44:55.227 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.107+0000 7f382d534700 1 -- start start 2024-02-05T07:44:55.227 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.107+0000 7f382d534700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f382807b730 0x7f3828079b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:55.227 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.107+0000 7f382d534700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382807a160 con 0x7f382807b730 2024-02-05T07:44:55.227 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.107+0000 7f3826ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f382807b730 0x7f3828079b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:55.227 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.107+0000 7f3826ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f382807b730 0x7f3828079b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54886/0 (socket says 172.21.15.131:54886) 2024-02-05T07:44:55.227 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.107+0000 7f3826ffd700 1 -- 172.21.15.131:0/2401386603 learned_addr learned my addr 172.21.15.131:0/2401386603 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:55.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.107+0000 7f3826ffd700 1 -- 172.21.15.131:0/2401386603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382807a2a0 con 0x7f382807b730 2024-02-05T07:44:55.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.107+0000 7f3826ffd700 1 --2- 172.21.15.131:0/2401386603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f382807b730 0x7f3828079b90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f38100098a0 tx=0x7f3810009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c505e010f59c5f server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:55.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.107+0000 7f3825ffb700 1 -- 172.21.15.131:0/2401386603 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3810004160 con 0x7f382807b730 2024-02-05T07:44:55.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.108+0000 7f3825ffb700 1 -- 172.21.15.131:0/2401386603 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f38100042c0 con 0x7f382807b730 2024-02-05T07:44:55.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.108+0000 7f3825ffb700 1 -- 172.21.15.131:0/2401386603 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38100119b0 con 0x7f382807b730 2024-02-05T07:44:55.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.108+0000 7f382d534700 1 -- 172.21.15.131:0/2401386603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f382807b730 msgr2=0x7f3828079b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:55.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.108+0000 7f382d534700 1 --2- 172.21.15.131:0/2401386603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f382807b730 0x7f3828079b90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f38100098a0 tx=0x7f3810009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:55.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.108+0000 7f382d534700 1 -- 172.21.15.131:0/2401386603 shutdown_connections 2024-02-05T07:44:55.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.108+0000 7f382d534700 1 --2- 172.21.15.131:0/2401386603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f382807b730 0x7f3828079b90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:55.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.108+0000 7f382d534700 1 -- 172.21.15.131:0/2401386603 >> 172.21.15.131:0/2401386603 conn(0x7f38281014d0 msgr2=0x7f38281038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:55.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.108+0000 7f382d534700 1 -- 172.21.15.131:0/2401386603 shutdown_connections 2024-02-05T07:44:55.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.108+0000 7f382d534700 1 -- 172.21.15.131:0/2401386603 wait complete. 2024-02-05T07:44:55.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.109+0000 7f382d534700 1 Processor -- start 2024-02-05T07:44:55.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.109+0000 7f382d534700 1 -- start start 2024-02-05T07:44:55.228 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.109+0000 7f382d534700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f382807b730 0x7f382819bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.109+0000 7f382d534700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382819c230 con 0x7f382807b730 2024-02-05T07:44:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.109+0000 7f3826ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f382807b730 0x7f382819bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.109+0000 7f3826ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f382807b730 0x7f382819bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54894/0 (socket says 172.21.15.131:54894) 2024-02-05T07:44:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.109+0000 7f3826ffd700 1 -- 172.21.15.131:0/1800613205 learned_addr learned my addr 172.21.15.131:0/1800613205 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.110+0000 7f3826ffd700 1 -- 172.21.15.131:0/1800613205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3810009550 con 0x7f382807b730 2024-02-05T07:44:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.110+0000 7f3826ffd700 1 --2- 172.21.15.131:0/1800613205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f382807b730 0x7f382819bcf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f38100046c0 tx=0x7f3810011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.110+0000 7f381ffff700 1 -- 172.21.15.131:0/1800613205 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38100038b0 con 0x7f382807b730 2024-02-05T07:44:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.110+0000 7f381ffff700 1 -- 172.21.15.131:0/1800613205 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f3810003a10 con 0x7f382807b730 2024-02-05T07:44:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.110+0000 7f382d534700 1 -- 172.21.15.131:0/1800613205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382819c430 con 0x7f382807b730 2024-02-05T07:44:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.110+0000 7f381ffff700 1 -- 172.21.15.131:0/1800613205 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3810019610 con 0x7f382807b730 2024-02-05T07:44:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.110+0000 7f382d534700 1 -- 172.21.15.131:0/1800613205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f382819c8d0 con 0x7f382807b730 2024-02-05T07:44:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.111+0000 7f381ffff700 1 -- 172.21.15.131:0/1800613205 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7f38100198d0 con 0x7f382807b730 2024-02-05T07:44:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.111+0000 7f382d534700 1 -- 172.21.15.131:0/1800613205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f382810c1b0 con 0x7f382807b730 2024-02-05T07:44:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.111+0000 7f381ffff700 1 --2- 172.21.15.131:0/1800613205 >> v2:172.21.15.131:6800/2947584178 conn(0x7f3814038e50 0x7f381403b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.111+0000 7f381ffff700 1 -- 172.21.15.131:0/1800613205 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f381004cfd0 con 0x7f382807b730 2024-02-05T07:44:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.111+0000 7f38267fc700 1 --2- 172.21.15.131:0/1800613205 >> v2:172.21.15.131:6800/2947584178 conn(0x7f3814038e50 0x7f381403b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:55.230 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.111+0000 7f38267fc700 1 --2- 172.21.15.131:0/1800613205 >> v2:172.21.15.131:6800/2947584178 conn(0x7f3814038e50 0x7f381403b310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3818006fd0 tx=0x7f3818006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:55.230 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.116+0000 7f381ffff700 1 -- 172.21.15.131:0/1800613205 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3810018350 con 0x7f382807b730 2024-02-05T07:44:55.230 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:50.310+0000 7f382d534700 1 -- 172.21.15.131:0/1800613205 --> v2:172.21.15.131:6800/2947584178 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi131", "addr": "172.21.15.131", "target": ["mon-mgr", ""]}) v1 -- 0x7f3828102740 con 0x7f3814038e50 2024-02-05T07:44:55.230 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:54.632+0000 7f381ffff700 1 -- 172.21.15.131:0/1800613205 <== mgr.14120 v2:172.21.15.131:6800/2947584178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f3828102740 con 0x7f3814038e50 2024-02-05T07:44:55.230 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:54.635+0000 7f382d534700 1 -- 172.21.15.131:0/1800613205 >> v2:172.21.15.131:6800/2947584178 conn(0x7f3814038e50 msgr2=0x7f381403b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:55.230 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:54.635+0000 7f382d534700 1 --2- 172.21.15.131:0/1800613205 >> v2:172.21.15.131:6800/2947584178 conn(0x7f3814038e50 0x7f381403b310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3818006fd0 tx=0x7f3818006e40 comp rx=0 tx=0).stop 2024-02-05T07:44:55.230 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:54.635+0000 7f382d534700 1 -- 172.21.15.131:0/1800613205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f382807b730 msgr2=0x7f382819bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:55.230 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:54.635+0000 7f382d534700 1 --2- 172.21.15.131:0/1800613205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f382807b730 0x7f382819bcf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f38100046c0 tx=0x7f3810011a50 comp rx=0 tx=0).stop 2024-02-05T07:44:55.230 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:54.635+0000 7f382d534700 1 -- 172.21.15.131:0/1800613205 shutdown_connections 2024-02-05T07:44:55.230 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:54.635+0000 7f382d534700 1 --2- 172.21.15.131:0/1800613205 >> v2:172.21.15.131:6800/2947584178 conn(0x7f3814038e50 0x7f381403b310 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:55.230 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:54.635+0000 7f382d534700 1 --2- 172.21.15.131:0/1800613205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f382807b730 0x7f382819bcf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:55.230 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:54.635+0000 7f382d534700 1 -- 172.21.15.131:0/1800613205 >> 172.21.15.131:0/1800613205 conn(0x7f38281014d0 msgr2=0x7f3828102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:55.230 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:54.636+0000 7f382d534700 1 -- 172.21.15.131:0/1800613205 shutdown_connections 2024-02-05T07:44:55.230 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:54.636+0000 7f382d534700 1 -- 172.21.15.131:0/1800613205 wait complete. 2024-02-05T07:44:55.230 INFO:teuthology.orchestra.run.smithi131.stdout:Deploying unmanaged mon service... 2024-02-05T07:44:55.745 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:55 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' 2024-02-05T07:44:55.745 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:55 smithi131 ceph-mon[29507]: Added host smithi131 2024-02-05T07:44:55.745 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:55 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:44:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:56 smithi131 ceph-mon[29507]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:44:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:56 smithi131 ceph-mon[29507]: Saving service mon spec with placement count:5 2024-02-05T07:44:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:56 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' 2024-02-05T07:44:57.566 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-05T07:44:57.567 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.744+0000 7f4d66f6c700 1 Processor -- start 2024-02-05T07:44:57.567 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.744+0000 7f4d66f6c700 1 -- start start 2024-02-05T07:44:57.567 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.744+0000 7f4d66f6c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d60105e60 0x7f4d60106280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:57.567 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.744+0000 7f4d66f6c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d601067c0 con 0x7f4d60105e60 2024-02-05T07:44:57.567 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.744+0000 7f4d64d08700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d60105e60 0x7f4d60106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:57.567 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.744+0000 7f4d64d08700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d60105e60 0x7f4d60106280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54896/0 (socket says 172.21.15.131:54896) 2024-02-05T07:44:57.567 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.744+0000 7f4d64d08700 1 -- 172.21.15.131:0/2638461407 learned_addr learned my addr 172.21.15.131:0/2638461407 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:57.567 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.745+0000 7f4d64d08700 1 -- 172.21.15.131:0/2638461407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d60106900 con 0x7f4d60105e60 2024-02-05T07:44:57.567 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.745+0000 7f4d64d08700 1 --2- 172.21.15.131:0/2638461407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d60105e60 0x7f4d60106280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4d540098a0 tx=0x7f4d54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a60a0c9b2364583 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:57.567 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.745+0000 7f4d5f7fe700 1 -- 172.21.15.131:0/2638461407 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d54004160 con 0x7f4d60105e60 2024-02-05T07:44:57.567 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.745+0000 7f4d5f7fe700 1 -- 172.21.15.131:0/2638461407 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f4d540042c0 con 0x7f4d60105e60 2024-02-05T07:44:57.567 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.745+0000 7f4d5f7fe700 1 -- 172.21.15.131:0/2638461407 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d540119b0 con 0x7f4d60105e60 2024-02-05T07:44:57.567 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.745+0000 7f4d66f6c700 1 -- 172.21.15.131:0/2638461407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d60105e60 msgr2=0x7f4d60106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:57.567 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.745+0000 7f4d66f6c700 1 --2- 172.21.15.131:0/2638461407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d60105e60 0x7f4d60106280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4d540098a0 tx=0x7f4d54009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:57.567 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.746+0000 7f4d66f6c700 1 -- 172.21.15.131:0/2638461407 shutdown_connections 2024-02-05T07:44:57.567 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.746+0000 7f4d66f6c700 1 --2- 172.21.15.131:0/2638461407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d60105e60 0x7f4d60106280 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:57.568 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.746+0000 7f4d66f6c700 1 -- 172.21.15.131:0/2638461407 >> 172.21.15.131:0/2638461407 conn(0x7f4d601014d0 msgr2=0x7f4d601038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:57.568 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.746+0000 7f4d66f6c700 1 -- 172.21.15.131:0/2638461407 shutdown_connections 2024-02-05T07:44:57.568 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.746+0000 7f4d66f6c700 1 -- 172.21.15.131:0/2638461407 wait complete. 2024-02-05T07:44:57.568 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.746+0000 7f4d66f6c700 1 Processor -- start 2024-02-05T07:44:57.568 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.746+0000 7f4d66f6c700 1 -- start start 2024-02-05T07:44:57.568 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.746+0000 7f4d66f6c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d60105e60 0x7f4d60199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:57.568 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.746+0000 7f4d66f6c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6019a050 con 0x7f4d60105e60 2024-02-05T07:44:57.568 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.746+0000 7f4d64d08700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d60105e60 0x7f4d60199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:57.568 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.746+0000 7f4d64d08700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d60105e60 0x7f4d60199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54908/0 (socket says 172.21.15.131:54908) 2024-02-05T07:44:57.568 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.746+0000 7f4d64d08700 1 -- 172.21.15.131:0/2627955539 learned_addr learned my addr 172.21.15.131:0/2627955539 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:57.568 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.747+0000 7f4d64d08700 1 -- 172.21.15.131:0/2627955539 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d54009550 con 0x7f4d60105e60 2024-02-05T07:44:57.568 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.747+0000 7f4d64d08700 1 --2- 172.21.15.131:0/2627955539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d60105e60 0x7f4d60199b10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4d54006af0 tx=0x7f4d54011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:57.568 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.747+0000 7f4d5dffb700 1 -- 172.21.15.131:0/2627955539 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d540039e0 con 0x7f4d60105e60 2024-02-05T07:44:57.568 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.747+0000 7f4d5dffb700 1 -- 172.21.15.131:0/2627955539 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f4d54003b40 con 0x7f4d60105e60 2024-02-05T07:44:57.568 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.747+0000 7f4d66f6c700 1 -- 172.21.15.131:0/2627955539 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6019a250 con 0x7f4d60105e60 2024-02-05T07:44:57.569 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.747+0000 7f4d5dffb700 1 -- 172.21.15.131:0/2627955539 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d54019700 con 0x7f4d60105e60 2024-02-05T07:44:57.569 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.747+0000 7f4d66f6c700 1 -- 172.21.15.131:0/2627955539 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6019a6f0 con 0x7f4d60105e60 2024-02-05T07:44:57.569 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.748+0000 7f4d5dffb700 1 -- 172.21.15.131:0/2627955539 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7f4d54019860 con 0x7f4d60105e60 2024-02-05T07:44:57.569 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.748+0000 7f4d66f6c700 1 -- 172.21.15.131:0/2627955539 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d60063350 con 0x7f4d60105e60 2024-02-05T07:44:57.569 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.748+0000 7f4d5dffb700 1 --2- 172.21.15.131:0/2627955539 >> v2:172.21.15.131:6800/2947584178 conn(0x7f4d50038ef0 0x7f4d5003b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:57.569 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.748+0000 7f4d5dffb700 1 -- 172.21.15.131:0/2627955539 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f4d5404d150 con 0x7f4d60105e60 2024-02-05T07:44:57.569 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.748+0000 7f4d5ffff700 1 --2- 172.21.15.131:0/2627955539 >> v2:172.21.15.131:6800/2947584178 conn(0x7f4d50038ef0 0x7f4d5003b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:57.569 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.749+0000 7f4d5ffff700 1 --2- 172.21.15.131:0/2627955539 >> v2:172.21.15.131:6800/2947584178 conn(0x7f4d50038ef0 0x7f4d5003b3b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c006fd0 tx=0x7f4d4c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:57.569 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.753+0000 7f4d5dffb700 1 -- 172.21.15.131:0/2627955539 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4d54019b10 con 0x7f4d60105e60 2024-02-05T07:44:57.569 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.973+0000 7f4d66f6c700 1 -- 172.21.15.131:0/2627955539 --> v2:172.21.15.131:6800/2947584178 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4d601027b0 con 0x7f4d50038ef0 2024-02-05T07:44:57.569 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.978+0000 7f4d5dffb700 1 -- 172.21.15.131:0/2627955539 <== mgr.14120 v2:172.21.15.131:6800/2947584178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4d601027b0 con 0x7f4d50038ef0 2024-02-05T07:44:57.569 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.980+0000 7f4d66f6c700 1 -- 172.21.15.131:0/2627955539 >> v2:172.21.15.131:6800/2947584178 conn(0x7f4d50038ef0 msgr2=0x7f4d5003b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:57.569 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.980+0000 7f4d66f6c700 1 --2- 172.21.15.131:0/2627955539 >> v2:172.21.15.131:6800/2947584178 conn(0x7f4d50038ef0 0x7f4d5003b3b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c006fd0 tx=0x7f4d4c006e40 comp rx=0 tx=0).stop 2024-02-05T07:44:57.569 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.980+0000 7f4d66f6c700 1 -- 172.21.15.131:0/2627955539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d60105e60 msgr2=0x7f4d60199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:57.569 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.980+0000 7f4d66f6c700 1 --2- 172.21.15.131:0/2627955539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d60105e60 0x7f4d60199b10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4d54006af0 tx=0x7f4d54011a50 comp rx=0 tx=0).stop 2024-02-05T07:44:57.570 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.980+0000 7f4d66f6c700 1 -- 172.21.15.131:0/2627955539 shutdown_connections 2024-02-05T07:44:57.570 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.980+0000 7f4d66f6c700 1 --2- 172.21.15.131:0/2627955539 >> v2:172.21.15.131:6800/2947584178 conn(0x7f4d50038ef0 0x7f4d5003b3b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:57.570 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.980+0000 7f4d66f6c700 1 --2- 172.21.15.131:0/2627955539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d60105e60 0x7f4d60199b10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:57.570 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.980+0000 7f4d66f6c700 1 -- 172.21.15.131:0/2627955539 >> 172.21.15.131:0/2627955539 conn(0x7f4d601014d0 msgr2=0x7f4d601020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:57.570 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.980+0000 7f4d66f6c700 1 -- 172.21.15.131:0/2627955539 shutdown_connections 2024-02-05T07:44:57.570 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:55.980+0000 7f4d66f6c700 1 -- 172.21.15.131:0/2627955539 wait complete. 2024-02-05T07:44:57.570 INFO:teuthology.orchestra.run.smithi131.stdout:Deploying unmanaged mgr service... 2024-02-05T07:44:58.097 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:58 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' 2024-02-05T07:44:58.097 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:58 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' 2024-02-05T07:44:59.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-05T07:44:59.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.199+0000 7f12cf7fb700 1 Processor -- start 2024-02-05T07:44:59.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.200+0000 7f12cf7fb700 1 -- start start 2024-02-05T07:44:59.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.200+0000 7f12cf7fb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12c8105e60 0x7f12c8106280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:59.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.200+0000 7f12cf7fb700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c81067c0 con 0x7f12c8105e60 2024-02-05T07:44:59.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.200+0000 7f12cd597700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12c8105e60 0x7f12c8106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:59.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.200+0000 7f12cd597700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12c8105e60 0x7f12c8106280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54920/0 (socket says 172.21.15.131:54920) 2024-02-05T07:44:59.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.200+0000 7f12cd597700 1 -- 172.21.15.131:0/281294124 learned_addr learned my addr 172.21.15.131:0/281294124 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:59.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.200+0000 7f12cd597700 1 -- 172.21.15.131:0/281294124 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c8106900 con 0x7f12c8105e60 2024-02-05T07:44:59.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.200+0000 7f12cd597700 1 --2- 172.21.15.131:0/281294124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12c8105e60 0x7f12c8106280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f12c40098a0 tx=0x7f12c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733aa8b05348d2f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:59.109 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.201+0000 7f12bffff700 1 -- 172.21.15.131:0/281294124 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12c4004160 con 0x7f12c8105e60 2024-02-05T07:44:59.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.201+0000 7f12bffff700 1 -- 172.21.15.131:0/281294124 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f12c40042c0 con 0x7f12c8105e60 2024-02-05T07:44:59.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.201+0000 7f12bffff700 1 -- 172.21.15.131:0/281294124 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12c4011960 con 0x7f12c8105e60 2024-02-05T07:44:59.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.201+0000 7f12cf7fb700 1 -- 172.21.15.131:0/281294124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12c8105e60 msgr2=0x7f12c8106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:59.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.201+0000 7f12cf7fb700 1 --2- 172.21.15.131:0/281294124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12c8105e60 0x7f12c8106280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f12c40098a0 tx=0x7f12c4009bb0 comp rx=0 tx=0).stop 2024-02-05T07:44:59.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.201+0000 7f12cf7fb700 1 -- 172.21.15.131:0/281294124 shutdown_connections 2024-02-05T07:44:59.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.201+0000 7f12cf7fb700 1 --2- 172.21.15.131:0/281294124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12c8105e60 0x7f12c8106280 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:59.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.201+0000 7f12cf7fb700 1 -- 172.21.15.131:0/281294124 >> 172.21.15.131:0/281294124 conn(0x7f12c81014d0 msgr2=0x7f12c81038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:59.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.201+0000 7f12cf7fb700 1 -- 172.21.15.131:0/281294124 shutdown_connections 2024-02-05T07:44:59.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.201+0000 7f12cf7fb700 1 -- 172.21.15.131:0/281294124 wait complete. 2024-02-05T07:44:59.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.202+0000 7f12cf7fb700 1 Processor -- start 2024-02-05T07:44:59.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.202+0000 7f12cf7fb700 1 -- start start 2024-02-05T07:44:59.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.202+0000 7f12cf7fb700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12c8105e60 0x7f12c8199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:59.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.202+0000 7f12cf7fb700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c819a0e0 con 0x7f12c8105e60 2024-02-05T07:44:59.110 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.202+0000 7f12cd597700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12c8105e60 0x7f12c8199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:59.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.202+0000 7f12cd597700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12c8105e60 0x7f12c8199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54926/0 (socket says 172.21.15.131:54926) 2024-02-05T07:44:59.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.202+0000 7f12cd597700 1 -- 172.21.15.131:0/4048038563 learned_addr learned my addr 172.21.15.131:0/4048038563 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:44:59.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.203+0000 7f12cd597700 1 -- 172.21.15.131:0/4048038563 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4009550 con 0x7f12c8105e60 2024-02-05T07:44:59.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.203+0000 7f12cd597700 1 --2- 172.21.15.131:0/4048038563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12c8105e60 0x7f12c8199ba0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f12c4003f60 tx=0x7f12c40119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:59.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.203+0000 7f12be7fc700 1 -- 172.21.15.131:0/4048038563 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12c4003970 con 0x7f12c8105e60 2024-02-05T07:44:59.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.203+0000 7f12be7fc700 1 -- 172.21.15.131:0/4048038563 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f12c4003ad0 con 0x7f12c8105e60 2024-02-05T07:44:59.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.203+0000 7f12cf7fb700 1 -- 172.21.15.131:0/4048038563 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c819a2e0 con 0x7f12c8105e60 2024-02-05T07:44:59.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.203+0000 7f12be7fc700 1 -- 172.21.15.131:0/4048038563 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12c4019720 con 0x7f12c8105e60 2024-02-05T07:44:59.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.203+0000 7f12cf7fb700 1 -- 172.21.15.131:0/4048038563 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c819a780 con 0x7f12c8105e60 2024-02-05T07:44:59.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.204+0000 7f12cf7fb700 1 -- 172.21.15.131:0/4048038563 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c8063350 con 0x7f12c8105e60 2024-02-05T07:44:59.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.204+0000 7f12be7fc700 1 -- 172.21.15.131:0/4048038563 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7f12c401a430 con 0x7f12c8105e60 2024-02-05T07:44:59.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.204+0000 7f12be7fc700 1 --2- 172.21.15.131:0/4048038563 >> v2:172.21.15.131:6800/2947584178 conn(0x7f12b4038ea0 0x7f12b403b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:44:59.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.205+0000 7f12be7fc700 1 -- 172.21.15.131:0/4048038563 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f12c404ce30 con 0x7f12c8105e60 2024-02-05T07:44:59.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.205+0000 7f12ccd96700 1 --2- 172.21.15.131:0/4048038563 >> v2:172.21.15.131:6800/2947584178 conn(0x7f12b4038ea0 0x7f12b403b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:44:59.111 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.205+0000 7f12ccd96700 1 --2- 172.21.15.131:0/4048038563 >> v2:172.21.15.131:6800/2947584178 conn(0x7f12b4038ea0 0x7f12b403b360 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f12b8006fd0 tx=0x7f12b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:44:59.112 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.209+0000 7f12be7fc700 1 -- 172.21.15.131:0/4048038563 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f12c4051070 con 0x7f12c8105e60 2024-02-05T07:44:59.112 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.436+0000 7f12cf7fb700 1 -- 172.21.15.131:0/4048038563 --> v2:172.21.15.131:6800/2947584178 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f12c81027e0 con 0x7f12b4038ea0 2024-02-05T07:44:59.112 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.443+0000 7f12be7fc700 1 -- 172.21.15.131:0/4048038563 <== mgr.14120 v2:172.21.15.131:6800/2947584178 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f12c81027e0 con 0x7f12b4038ea0 2024-02-05T07:44:59.112 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.445+0000 7f12cf7fb700 1 -- 172.21.15.131:0/4048038563 >> v2:172.21.15.131:6800/2947584178 conn(0x7f12b4038ea0 msgr2=0x7f12b403b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:59.112 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.445+0000 7f12cf7fb700 1 --2- 172.21.15.131:0/4048038563 >> v2:172.21.15.131:6800/2947584178 conn(0x7f12b4038ea0 0x7f12b403b360 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f12b8006fd0 tx=0x7f12b8006e40 comp rx=0 tx=0).stop 2024-02-05T07:44:59.112 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.445+0000 7f12cf7fb700 1 -- 172.21.15.131:0/4048038563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12c8105e60 msgr2=0x7f12c8199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:44:59.112 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.445+0000 7f12cf7fb700 1 --2- 172.21.15.131:0/4048038563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12c8105e60 0x7f12c8199ba0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f12c4003f60 tx=0x7f12c40119e0 comp rx=0 tx=0).stop 2024-02-05T07:44:59.112 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.445+0000 7f12cf7fb700 1 -- 172.21.15.131:0/4048038563 shutdown_connections 2024-02-05T07:44:59.112 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.445+0000 7f12cf7fb700 1 --2- 172.21.15.131:0/4048038563 >> v2:172.21.15.131:6800/2947584178 conn(0x7f12b4038ea0 0x7f12b403b360 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:59.112 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.445+0000 7f12cf7fb700 1 --2- 172.21.15.131:0/4048038563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12c8105e60 0x7f12c8199ba0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:44:59.112 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.445+0000 7f12cf7fb700 1 -- 172.21.15.131:0/4048038563 >> 172.21.15.131:0/4048038563 conn(0x7f12c81014d0 msgr2=0x7f12c81020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:44:59.112 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.445+0000 7f12cf7fb700 1 -- 172.21.15.131:0/4048038563 shutdown_connections 2024-02-05T07:44:59.112 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:58.445+0000 7f12cf7fb700 1 -- 172.21.15.131:0/4048038563 wait complete. 2024-02-05T07:44:59.744 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:59 smithi131 ceph-mon[29507]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:44:59.744 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:59 smithi131 ceph-mon[29507]: Saving service mgr spec with placement count:2 2024-02-05T07:44:59.744 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:44:59 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' 2024-02-05T07:45:00.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.541+0000 7f74c7366700 1 Processor -- start 2024-02-05T07:45:00.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.541+0000 7f74c7366700 1 -- start start 2024-02-05T07:45:00.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.541+0000 7f74c7366700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c0105e60 0x7f74c0106280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:00.499 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.541+0000 7f74c7366700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c01067c0 con 0x7f74c0105e60 2024-02-05T07:45:00.500 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.541+0000 7f74c5102700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c0105e60 0x7f74c0106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:00.500 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.541+0000 7f74c5102700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c0105e60 0x7f74c0106280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34690/0 (socket says 172.21.15.131:34690) 2024-02-05T07:45:00.500 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.541+0000 7f74c5102700 1 -- 172.21.15.131:0/3636713308 learned_addr learned my addr 172.21.15.131:0/3636713308 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:00.500 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.542+0000 7f74c5102700 1 -- 172.21.15.131:0/3636713308 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c0106900 con 0x7f74c0105e60 2024-02-05T07:45:00.500 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.542+0000 7f74c5102700 1 --2- 172.21.15.131:0/3636713308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c0105e60 0x7f74c0106280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0098a0 tx=0x7f74bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2a96b4af6ef056 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:00.500 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.542+0000 7f74b7fff700 1 -- 172.21.15.131:0/3636713308 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74bc004160 con 0x7f74c0105e60 2024-02-05T07:45:00.500 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.542+0000 7f74b7fff700 1 -- 172.21.15.131:0/3636713308 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f74bc0042c0 con 0x7f74c0105e60 2024-02-05T07:45:00.500 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.542+0000 7f74b7fff700 1 -- 172.21.15.131:0/3636713308 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74bc011960 con 0x7f74c0105e60 2024-02-05T07:45:00.500 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.542+0000 7f74c7366700 1 -- 172.21.15.131:0/3636713308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c0105e60 msgr2=0x7f74c0106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:00.500 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.542+0000 7f74c7366700 1 --2- 172.21.15.131:0/3636713308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c0105e60 0x7f74c0106280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0098a0 tx=0x7f74bc009bb0 comp rx=0 tx=0).stop 2024-02-05T07:45:00.500 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.542+0000 7f74c7366700 1 -- 172.21.15.131:0/3636713308 shutdown_connections 2024-02-05T07:45:00.500 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.542+0000 7f74c7366700 1 --2- 172.21.15.131:0/3636713308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c0105e60 0x7f74c0106280 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:00.500 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.542+0000 7f74c7366700 1 -- 172.21.15.131:0/3636713308 >> 172.21.15.131:0/3636713308 conn(0x7f74c01014d0 msgr2=0x7f74c01038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.543+0000 7f74c7366700 1 -- 172.21.15.131:0/3636713308 shutdown_connections 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.543+0000 7f74c7366700 1 -- 172.21.15.131:0/3636713308 wait complete. 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.543+0000 7f74c7366700 1 Processor -- start 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.543+0000 7f74c7366700 1 -- start start 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.543+0000 7f74c7366700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c0105e60 0x7f74c0199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.543+0000 7f74c7366700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c019a070 con 0x7f74c0105e60 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.543+0000 7f74c5102700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c0105e60 0x7f74c0199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.543+0000 7f74c5102700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c0105e60 0x7f74c0199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34706/0 (socket says 172.21.15.131:34706) 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.543+0000 7f74c5102700 1 -- 172.21.15.131:0/1048455799 learned_addr learned my addr 172.21.15.131:0/1048455799 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.544+0000 7f74c5102700 1 -- 172.21.15.131:0/1048455799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc009550 con 0x7f74c0105e60 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.544+0000 7f74c5102700 1 --2- 172.21.15.131:0/1048455799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c0105e60 0x7f74c0199b30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f74bc006af0 tx=0x7f74bc011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.544+0000 7f74b67fc700 1 -- 172.21.15.131:0/1048455799 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74bc0036a0 con 0x7f74c0105e60 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.544+0000 7f74b67fc700 1 -- 172.21.15.131:0/1048455799 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f74bc01a430 con 0x7f74c0105e60 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.544+0000 7f74c7366700 1 -- 172.21.15.131:0/1048455799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c019a270 con 0x7f74c0105e60 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.544+0000 7f74b67fc700 1 -- 172.21.15.131:0/1048455799 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74bc0193c0 con 0x7f74c0105e60 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.544+0000 7f74c7366700 1 -- 172.21.15.131:0/1048455799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c019a710 con 0x7f74c0105e60 2024-02-05T07:45:00.501 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.545+0000 7f74c7366700 1 -- 172.21.15.131:0/1048455799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c0063350 con 0x7f74c0105e60 2024-02-05T07:45:00.502 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.545+0000 7f74b67fc700 1 -- 172.21.15.131:0/1048455799 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7f74bc01a880 con 0x7f74c0105e60 2024-02-05T07:45:00.502 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.545+0000 7f74b67fc700 1 --2- 172.21.15.131:0/1048455799 >> v2:172.21.15.131:6800/2947584178 conn(0x7f74ac038ef0 0x7f74ac03b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:00.502 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.545+0000 7f74b67fc700 1 -- 172.21.15.131:0/1048455799 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f74bc04cab0 con 0x7f74c0105e60 2024-02-05T07:45:00.502 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.545+0000 7f74c4901700 1 --2- 172.21.15.131:0/1048455799 >> v2:172.21.15.131:6800/2947584178 conn(0x7f74ac038ef0 0x7f74ac03b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:00.502 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.546+0000 7f74c4901700 1 --2- 172.21.15.131:0/1048455799 >> v2:172.21.15.131:6800/2947584178 conn(0x7f74ac038ef0 0x7f74ac03b3b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f74b0006fd0 tx=0x7f74b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:00.502 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.550+0000 7f74b67fc700 1 -- 172.21.15.131:0/1048455799 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f74bc01a5a0 con 0x7f74c0105e60 2024-02-05T07:45:00.502 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.742+0000 7f74c7366700 1 -- 172.21.15.131:0/1048455799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f74c019cff0 con 0x7f74c0105e60 2024-02-05T07:45:00.502 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.745+0000 7f74b67fc700 1 -- 172.21.15.131:0/1048455799 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f74bc02b350 con 0x7f74c0105e60 2024-02-05T07:45:00.502 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.748+0000 7f74c7366700 1 -- 172.21.15.131:0/1048455799 >> v2:172.21.15.131:6800/2947584178 conn(0x7f74ac038ef0 msgr2=0x7f74ac03b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:00.502 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.748+0000 7f74c7366700 1 --2- 172.21.15.131:0/1048455799 >> v2:172.21.15.131:6800/2947584178 conn(0x7f74ac038ef0 0x7f74ac03b3b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f74b0006fd0 tx=0x7f74b0006e40 comp rx=0 tx=0).stop 2024-02-05T07:45:00.502 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.748+0000 7f74c7366700 1 -- 172.21.15.131:0/1048455799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c0105e60 msgr2=0x7f74c0199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:00.502 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.748+0000 7f74c7366700 1 --2- 172.21.15.131:0/1048455799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c0105e60 0x7f74c0199b30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f74bc006af0 tx=0x7f74bc011eb0 comp rx=0 tx=0).stop 2024-02-05T07:45:00.502 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.748+0000 7f74c7366700 1 -- 172.21.15.131:0/1048455799 shutdown_connections 2024-02-05T07:45:00.502 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.748+0000 7f74c7366700 1 --2- 172.21.15.131:0/1048455799 >> v2:172.21.15.131:6800/2947584178 conn(0x7f74ac038ef0 0x7f74ac03b3b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:00.502 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.748+0000 7f74c7366700 1 --2- 172.21.15.131:0/1048455799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74c0105e60 0x7f74c0199b30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:00.502 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.748+0000 7f74c7366700 1 -- 172.21.15.131:0/1048455799 >> 172.21.15.131:0/1048455799 conn(0x7f74c01014d0 msgr2=0x7f74c0101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:00.503 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.748+0000 7f74c7366700 1 -- 172.21.15.131:0/1048455799 shutdown_connections 2024-02-05T07:45:00.503 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:44:59.748+0000 7f74c7366700 1 -- 172.21.15.131:0/1048455799 wait complete. 2024-02-05T07:45:00.775 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1048455799' entity='client.admin' 2024-02-05T07:45:00.775 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:00 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' 2024-02-05T07:45:01.848 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.019+0000 7f80520c6700 1 Processor -- start 2024-02-05T07:45:01.849 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.019+0000 7f80520c6700 1 -- start start 2024-02-05T07:45:01.849 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.019+0000 7f80520c6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f804c105200 0x7f804c105620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:01.849 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.019+0000 7f80520c6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c105bf0 con 0x7f804c105200 2024-02-05T07:45:01.849 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.020+0000 7f804b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f804c105200 0x7f804c105620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:01.849 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.020+0000 7f804b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f804c105200 0x7f804c105620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34718/0 (socket says 172.21.15.131:34718) 2024-02-05T07:45:01.849 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.020+0000 7f804b7fe700 1 -- 172.21.15.131:0/4149634573 learned_addr learned my addr 172.21.15.131:0/4149634573 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:01.849 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.020+0000 7f804b7fe700 1 -- 172.21.15.131:0/4149634573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c105f20 con 0x7f804c105200 2024-02-05T07:45:01.849 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.020+0000 7f804b7fe700 1 --2- 172.21.15.131:0/4149634573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f804c105200 0x7f804c105620 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80400098a0 tx=0x7f8040009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3c586b5eb154b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:01.849 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.021+0000 7f804a7fc700 1 -- 172.21.15.131:0/4149634573 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8040004160 con 0x7f804c105200 2024-02-05T07:45:01.849 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.021+0000 7f804a7fc700 1 -- 172.21.15.131:0/4149634573 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f80400042c0 con 0x7f804c105200 2024-02-05T07:45:01.849 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.021+0000 7f804a7fc700 1 -- 172.21.15.131:0/4149634573 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8040011960 con 0x7f804c105200 2024-02-05T07:45:01.849 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.021+0000 7f80520c6700 1 -- 172.21.15.131:0/4149634573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f804c105200 msgr2=0x7f804c105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:01.849 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.021+0000 7f80520c6700 1 --2- 172.21.15.131:0/4149634573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f804c105200 0x7f804c105620 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80400098a0 tx=0x7f8040009bb0 comp rx=0 tx=0).stop 2024-02-05T07:45:01.850 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.021+0000 7f80520c6700 1 -- 172.21.15.131:0/4149634573 shutdown_connections 2024-02-05T07:45:01.850 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.021+0000 7f80520c6700 1 --2- 172.21.15.131:0/4149634573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f804c105200 0x7f804c105620 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:01.850 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.021+0000 7f80520c6700 1 -- 172.21.15.131:0/4149634573 >> 172.21.15.131:0/4149634573 conn(0x7f804c0ffa40 msgr2=0x7f804c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:01.850 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.021+0000 7f80520c6700 1 -- 172.21.15.131:0/4149634573 shutdown_connections 2024-02-05T07:45:01.850 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.021+0000 7f80520c6700 1 -- 172.21.15.131:0/4149634573 wait complete. 2024-02-05T07:45:01.850 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.022+0000 7f80520c6700 1 Processor -- start 2024-02-05T07:45:01.850 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.022+0000 7f80520c6700 1 -- start start 2024-02-05T07:45:01.850 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.022+0000 7f80520c6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f804c105200 0x7f804c18fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:01.850 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.022+0000 7f80520c6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c1902b0 con 0x7f804c105200 2024-02-05T07:45:01.850 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.023+0000 7f804b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f804c105200 0x7f804c18fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:01.850 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.023+0000 7f804b7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f804c105200 0x7f804c18fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34726/0 (socket says 172.21.15.131:34726) 2024-02-05T07:45:01.850 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.023+0000 7f804b7fe700 1 -- 172.21.15.131:0/979360481 learned_addr learned my addr 172.21.15.131:0/979360481 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:01.850 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.023+0000 7f804b7fe700 1 -- 172.21.15.131:0/979360481 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8040009550 con 0x7f804c105200 2024-02-05T07:45:01.850 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.023+0000 7f804b7fe700 1 --2- 172.21.15.131:0/979360481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f804c105200 0x7f804c18fd70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8040006af0 tx=0x7f80400119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:01.850 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.023+0000 7f8048ff9700 1 -- 172.21.15.131:0/979360481 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80400038b0 con 0x7f804c105200 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.023+0000 7f8048ff9700 1 -- 172.21.15.131:0/979360481 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f8040003a10 con 0x7f804c105200 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.023+0000 7f8048ff9700 1 -- 172.21.15.131:0/979360481 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8040019610 con 0x7f804c105200 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.023+0000 7f80520c6700 1 -- 172.21.15.131:0/979360481 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804c18e4a0 con 0x7f804c105200 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.023+0000 7f80520c6700 1 -- 172.21.15.131:0/979360481 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804c18e940 con 0x7f804c105200 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.024+0000 7f8048ff9700 1 -- 172.21.15.131:0/979360481 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7f8040019840 con 0x7f804c105200 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.024+0000 7f8048ff9700 1 --2- 172.21.15.131:0/979360481 >> v2:172.21.15.131:6800/2947584178 conn(0x7f8034038ea0 0x7f803403b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.024+0000 7f8048ff9700 1 -- 172.21.15.131:0/979360481 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f804004ce10 con 0x7f804c105200 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.025+0000 7f804affd700 1 --2- 172.21.15.131:0/979360481 >> v2:172.21.15.131:6800/2947584178 conn(0x7f8034038ea0 0x7f803403b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.025+0000 7f80520c6700 1 -- 172.21.15.131:0/979360481 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804c18f350 con 0x7f804c105200 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.025+0000 7f804affd700 1 --2- 172.21.15.131:0/979360481 >> v2:172.21.15.131:6800/2947584178 conn(0x7f8034038ea0 0x7f803403b360 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f803c006fd0 tx=0x7f803c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.030+0000 7f8048ff9700 1 -- 172.21.15.131:0/979360481 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f804002b340 con 0x7f804c105200 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.273+0000 7f80520c6700 1 -- 172.21.15.131:0/979360481 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f804c1093f0 con 0x7f804c105200 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.276+0000 7f8048ff9700 1 -- 172.21.15.131:0/979360481 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f8040018be0 con 0x7f804c105200 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.278+0000 7f80520c6700 1 -- 172.21.15.131:0/979360481 >> v2:172.21.15.131:6800/2947584178 conn(0x7f8034038ea0 msgr2=0x7f803403b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.278+0000 7f80520c6700 1 --2- 172.21.15.131:0/979360481 >> v2:172.21.15.131:6800/2947584178 conn(0x7f8034038ea0 0x7f803403b360 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f803c006fd0 tx=0x7f803c006e40 comp rx=0 tx=0).stop 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.279+0000 7f80520c6700 1 -- 172.21.15.131:0/979360481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f804c105200 msgr2=0x7f804c18fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:01.851 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.279+0000 7f80520c6700 1 --2- 172.21.15.131:0/979360481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f804c105200 0x7f804c18fd70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8040006af0 tx=0x7f80400119c0 comp rx=0 tx=0).stop 2024-02-05T07:45:01.852 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.279+0000 7f80520c6700 1 -- 172.21.15.131:0/979360481 shutdown_connections 2024-02-05T07:45:01.852 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.279+0000 7f80520c6700 1 --2- 172.21.15.131:0/979360481 >> v2:172.21.15.131:6800/2947584178 conn(0x7f8034038ea0 0x7f803403b360 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:01.852 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.279+0000 7f80520c6700 1 --2- 172.21.15.131:0/979360481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f804c105200 0x7f804c18fd70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:01.852 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.279+0000 7f80520c6700 1 -- 172.21.15.131:0/979360481 >> 172.21.15.131:0/979360481 conn(0x7f804c0ffa40 msgr2=0x7f804c1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:01.852 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.279+0000 7f80520c6700 1 -- 172.21.15.131:0/979360481 shutdown_connections 2024-02-05T07:45:01.852 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:01.279+0000 7f80520c6700 1 -- 172.21.15.131:0/979360481 wait complete. 2024-02-05T07:45:01.852 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling the dashboard module... 2024-02-05T07:45:02.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:02 smithi131 ceph-mon[29507]: from='mgr.14120 172.21.15.131:0/646944304' entity='mgr.y' 2024-02-05T07:45:02.496 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/979360481' entity='client.admin' 2024-02-05T07:45:03.986 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:03 smithi131 ceph-mon[29507]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:03.986 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/393363482' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-05T07:45:03.987 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:03 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ignoring --setuser ceph since I am not root 2024-02-05T07:45:03.987 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:03 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ignoring --setgroup ceph since I am not root 2024-02-05T07:45:04.286 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.516+0000 7f38c967e700 1 Processor -- start 2024-02-05T07:45:04.286 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.516+0000 7f38c967e700 1 -- start start 2024-02-05T07:45:04.286 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.516+0000 7f38c967e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c41068b0 0x7f38c4108ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:04.286 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.516+0000 7f38c967e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c4079740 con 0x7f38c41068b0 2024-02-05T07:45:04.286 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.516+0000 7f38c2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c41068b0 0x7f38c4108ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:04.286 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.516+0000 7f38c2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c41068b0 0x7f38c4108ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34734/0 (socket says 172.21.15.131:34734) 2024-02-05T07:45:04.286 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.516+0000 7f38c2ffd700 1 -- 172.21.15.131:0/1763935885 learned_addr learned my addr 172.21.15.131:0/1763935885 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:04.286 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.516+0000 7f38c2ffd700 1 -- 172.21.15.131:0/1763935885 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c41091e0 con 0x7f38c41068b0 2024-02-05T07:45:04.286 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.517+0000 7f38c2ffd700 1 --2- 172.21.15.131:0/1763935885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c41068b0 0x7f38c4108ca0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f38b40098a0 tx=0x7f38b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2230f3ceef26924 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:04.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.517+0000 7f38c1ffb700 1 -- 172.21.15.131:0/1763935885 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38b4004160 con 0x7f38c41068b0 2024-02-05T07:45:04.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.517+0000 7f38c1ffb700 1 -- 172.21.15.131:0/1763935885 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f38b40042c0 con 0x7f38c41068b0 2024-02-05T07:45:04.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.517+0000 7f38c1ffb700 1 -- 172.21.15.131:0/1763935885 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38b4011960 con 0x7f38c41068b0 2024-02-05T07:45:04.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.517+0000 7f38c967e700 1 -- 172.21.15.131:0/1763935885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c41068b0 msgr2=0x7f38c4108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:04.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.517+0000 7f38c967e700 1 --2- 172.21.15.131:0/1763935885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c41068b0 0x7f38c4108ca0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f38b40098a0 tx=0x7f38b4009bb0 comp rx=0 tx=0).stop 2024-02-05T07:45:04.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.518+0000 7f38c967e700 1 -- 172.21.15.131:0/1763935885 shutdown_connections 2024-02-05T07:45:04.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.518+0000 7f38c967e700 1 --2- 172.21.15.131:0/1763935885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c41068b0 0x7f38c4108ca0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:04.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.518+0000 7f38c967e700 1 -- 172.21.15.131:0/1763935885 >> 172.21.15.131:0/1763935885 conn(0x7f38c41002a0 msgr2=0x7f38c4102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:04.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.518+0000 7f38c967e700 1 -- 172.21.15.131:0/1763935885 shutdown_connections 2024-02-05T07:45:04.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.518+0000 7f38c967e700 1 -- 172.21.15.131:0/1763935885 wait complete. 2024-02-05T07:45:04.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.518+0000 7f38c967e700 1 Processor -- start 2024-02-05T07:45:04.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.518+0000 7f38c967e700 1 -- start start 2024-02-05T07:45:04.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.518+0000 7f38c967e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c41068b0 0x7f38c4106130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:04.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.518+0000 7f38c967e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c4106670 con 0x7f38c41068b0 2024-02-05T07:45:04.287 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.519+0000 7f38c2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c41068b0 0x7f38c4106130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:04.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.519+0000 7f38c2ffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c41068b0 0x7f38c4106130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34746/0 (socket says 172.21.15.131:34746) 2024-02-05T07:45:04.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.519+0000 7f38c2ffd700 1 -- 172.21.15.131:0/393363482 learned_addr learned my addr 172.21.15.131:0/393363482 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:04.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.519+0000 7f38c2ffd700 1 -- 172.21.15.131:0/393363482 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b4009550 con 0x7f38c41068b0 2024-02-05T07:45:04.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.519+0000 7f38c2ffd700 1 --2- 172.21.15.131:0/393363482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c41068b0 0x7f38c4106130 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f38b4006af0 tx=0x7f38b40119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:04.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.519+0000 7f38abfff700 1 -- 172.21.15.131:0/393363482 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38b40038b0 con 0x7f38c41068b0 2024-02-05T07:45:04.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.519+0000 7f38abfff700 1 -- 172.21.15.131:0/393363482 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f38b4003a10 con 0x7f38c41068b0 2024-02-05T07:45:04.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.519+0000 7f38c967e700 1 -- 172.21.15.131:0/393363482 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c4104860 con 0x7f38c41068b0 2024-02-05T07:45:04.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.519+0000 7f38abfff700 1 -- 172.21.15.131:0/393363482 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38b4019610 con 0x7f38c41068b0 2024-02-05T07:45:04.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.519+0000 7f38c967e700 1 -- 172.21.15.131:0/393363482 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c4104d00 con 0x7f38c41068b0 2024-02-05T07:45:04.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.520+0000 7f38abfff700 1 -- 172.21.15.131:0/393363482 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7f38b4019890 con 0x7f38c41068b0 2024-02-05T07:45:04.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.521+0000 7f38abfff700 1 --2- 172.21.15.131:0/393363482 >> v2:172.21.15.131:6800/2947584178 conn(0x7f38ac038e50 0x7f38ac03b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:04.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.521+0000 7f38c967e700 1 -- 172.21.15.131:0/393363482 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c410a020 con 0x7f38c41068b0 2024-02-05T07:45:04.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.521+0000 7f38abfff700 1 -- 172.21.15.131:0/393363482 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f38b404dde0 con 0x7f38c41068b0 2024-02-05T07:45:04.288 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.521+0000 7f38c27fc700 1 --2- 172.21.15.131:0/393363482 >> v2:172.21.15.131:6800/2947584178 conn(0x7f38ac038e50 0x7f38ac03b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:04.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.521+0000 7f38c27fc700 1 --2- 172.21.15.131:0/393363482 >> v2:172.21.15.131:6800/2947584178 conn(0x7f38ac038e50 0x7f38ac03b310 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f38b8006fd0 tx=0x7f38b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:04.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.526+0000 7f38abfff700 1 -- 172.21.15.131:0/393363482 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f38b402b340 con 0x7f38c41068b0 2024-02-05T07:45:04.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:02.777+0000 7f38c967e700 1 -- 172.21.15.131:0/393363482 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f38c410a230 con 0x7f38c41068b0 2024-02-05T07:45:04.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:03.663+0000 7f38abfff700 1 -- 172.21.15.131:0/393363482 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f38b4018be0 con 0x7f38c41068b0 2024-02-05T07:45:04.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:03.665+0000 7f38abfff700 1 -- 172.21.15.131:0/393363482 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mgrmap(e 9) v1 ==== 45910+0+0 (secure 0 0 0) 0x7f38b402d960 con 0x7f38c41068b0 2024-02-05T07:45:04.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:03.668+0000 7f38c967e700 1 -- 172.21.15.131:0/393363482 >> v2:172.21.15.131:6800/2947584178 conn(0x7f38ac038e50 msgr2=0x7f38ac03b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:04.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:03.668+0000 7f38c967e700 1 --2- 172.21.15.131:0/393363482 >> v2:172.21.15.131:6800/2947584178 conn(0x7f38ac038e50 0x7f38ac03b310 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f38b8006fd0 tx=0x7f38b8006e40 comp rx=0 tx=0).stop 2024-02-05T07:45:04.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:03.668+0000 7f38c967e700 1 -- 172.21.15.131:0/393363482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c41068b0 msgr2=0x7f38c4106130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:04.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:03.668+0000 7f38c967e700 1 --2- 172.21.15.131:0/393363482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c41068b0 0x7f38c4106130 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f38b4006af0 tx=0x7f38b40119c0 comp rx=0 tx=0).stop 2024-02-05T07:45:04.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:03.668+0000 7f38c967e700 1 -- 172.21.15.131:0/393363482 shutdown_connections 2024-02-05T07:45:04.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:03.668+0000 7f38c967e700 1 --2- 172.21.15.131:0/393363482 >> v2:172.21.15.131:6800/2947584178 conn(0x7f38ac038e50 0x7f38ac03b310 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:04.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:03.668+0000 7f38c967e700 1 --2- 172.21.15.131:0/393363482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38c41068b0 0x7f38c4106130 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:04.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:03.668+0000 7f38c967e700 1 -- 172.21.15.131:0/393363482 >> 172.21.15.131:0/393363482 conn(0x7f38c41002a0 msgr2=0x7f38c4100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:04.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:03.668+0000 7f38c967e700 1 -- 172.21.15.131:0/393363482 shutdown_connections 2024-02-05T07:45:04.289 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:03.668+0000 7f38c967e700 1 -- 172.21.15.131:0/393363482 wait complete. 2024-02-05T07:45:04.517 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:04 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:04.239+0000 7f9d1ca54200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T07:45:04.517 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:04 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:04.337+0000 7f9d1ca54200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T07:45:04.811 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/393363482' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-05T07:45:04.811 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:04 smithi131 ceph-mon[29507]: mgrmap e9: y(active, since 21s) 2024-02-05T07:45:05.078 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:05 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:05.003+0000 7f9d1ca54200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T07:45:05.347 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:05 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:05.102+0000 7f9d1ca54200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T07:45:05.347 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:05 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:05.248+0000 7f9d1ca54200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T07:45:05.348 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:05 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:05.346+0000 7f9d1ca54200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T07:45:05.642 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:05 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:05.431+0000 7f9d1ca54200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T07:45:05.651 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-02-05T07:45:05.651 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-05T07:45:05.651 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-05T07:45:05.651 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-05T07:45:05.651 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-05T07:45:05.651 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-02-05T07:45:05.651 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.810+0000 7f5d14024700 1 Processor -- start 2024-02-05T07:45:05.652 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.810+0000 7f5d14024700 1 -- start start 2024-02-05T07:45:05.652 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.811+0000 7f5d14024700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d0c104d60 0x7f5d0c105180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:05.652 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.811+0000 7f5d14024700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c105750 con 0x7f5d0c104d60 2024-02-05T07:45:05.652 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.811+0000 7f5d11dc0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d0c104d60 0x7f5d0c105180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:05.652 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.811+0000 7f5d11dc0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d0c104d60 0x7f5d0c105180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34766/0 (socket says 172.21.15.131:34766) 2024-02-05T07:45:05.652 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.811+0000 7f5d11dc0700 1 -- 172.21.15.131:0/1148215970 learned_addr learned my addr 172.21.15.131:0/1148215970 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:05.652 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.811+0000 7f5d11dc0700 1 -- 172.21.15.131:0/1148215970 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c105f60 con 0x7f5d0c104d60 2024-02-05T07:45:05.652 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.811+0000 7f5d11dc0700 1 --2- 172.21.15.131:0/1148215970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d0c104d60 0x7f5d0c105180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800bb40 tx=0x7f5d0800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c721862ab75be488 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:05.652 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.811+0000 7f5d10dbe700 1 -- 172.21.15.131:0/1148215970 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d0800dbe0 con 0x7f5d0c104d60 2024-02-05T07:45:05.652 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.811+0000 7f5d10dbe700 1 -- 172.21.15.131:0/1148215970 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f5d080044e0 con 0x7f5d0c104d60 2024-02-05T07:45:05.652 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.811+0000 7f5d10dbe700 1 -- 172.21.15.131:0/1148215970 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d08003e20 con 0x7f5d0c104d60 2024-02-05T07:45:05.652 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.812+0000 7f5d14024700 1 -- 172.21.15.131:0/1148215970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d0c104d60 msgr2=0x7f5d0c105180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:05.652 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.812+0000 7f5d14024700 1 --2- 172.21.15.131:0/1148215970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d0c104d60 0x7f5d0c105180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800bb40 tx=0x7f5d0800be50 comp rx=0 tx=0).stop 2024-02-05T07:45:05.652 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.812+0000 7f5d14024700 1 -- 172.21.15.131:0/1148215970 shutdown_connections 2024-02-05T07:45:05.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.812+0000 7f5d14024700 1 --2- 172.21.15.131:0/1148215970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d0c104d60 0x7f5d0c105180 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:05.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.812+0000 7f5d14024700 1 -- 172.21.15.131:0/1148215970 >> 172.21.15.131:0/1148215970 conn(0x7f5d0c100300 msgr2=0x7f5d0c102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:05.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.812+0000 7f5d14024700 1 -- 172.21.15.131:0/1148215970 shutdown_connections 2024-02-05T07:45:05.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.812+0000 7f5d14024700 1 -- 172.21.15.131:0/1148215970 wait complete. 2024-02-05T07:45:05.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.813+0000 7f5d14024700 1 Processor -- start 2024-02-05T07:45:05.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.813+0000 7f5d14024700 1 -- start start 2024-02-05T07:45:05.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.813+0000 7f5d14024700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d0c104d60 0x7f5d0c19bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:05.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.813+0000 7f5d14024700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c19c310 con 0x7f5d0c104d60 2024-02-05T07:45:05.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.813+0000 7f5d11dc0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d0c104d60 0x7f5d0c19bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:05.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.813+0000 7f5d11dc0700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d0c104d60 0x7f5d0c19bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34776/0 (socket says 172.21.15.131:34776) 2024-02-05T07:45:05.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.813+0000 7f5d11dc0700 1 -- 172.21.15.131:0/393646634 learned_addr learned my addr 172.21.15.131:0/393646634 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:05.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.813+0000 7f5d11dc0700 1 -- 172.21.15.131:0/393646634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0800b7f0 con 0x7f5d0c104d60 2024-02-05T07:45:05.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.813+0000 7f5d11dc0700 1 --2- 172.21.15.131:0/393646634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d0c104d60 0x7f5d0c19bdd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800d650 tx=0x7f5d080039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:05.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.813+0000 7f5d02ffd700 1 -- 172.21.15.131:0/393646634 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d08010630 con 0x7f5d0c104d60 2024-02-05T07:45:05.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.813+0000 7f5d02ffd700 1 -- 172.21.15.131:0/393646634 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f5d08010c40 con 0x7f5d0c104d60 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.813+0000 7f5d14024700 1 -- 172.21.15.131:0/393646634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0c19c510 con 0x7f5d0c104d60 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.814+0000 7f5d02ffd700 1 -- 172.21.15.131:0/393646634 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d08019b90 con 0x7f5d0c104d60 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.814+0000 7f5d14024700 1 -- 172.21.15.131:0/393646634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c19c950 con 0x7f5d0c104d60 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.814+0000 7f5d02ffd700 1 -- 172.21.15.131:0/393646634 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 9) v1 ==== 45910+0+0 (secure 0 0 0) 0x7f5d0802a450 con 0x7f5d0c104d60 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.814+0000 7f5d14024700 1 -- 172.21.15.131:0/393646634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c1915f0 con 0x7f5d0c104d60 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.814+0000 7f5d02ffd700 1 --2- 172.21.15.131:0/393646634 >> v2:172.21.15.131:6800/2947584178 conn(0x7f5cf8038ef0 0x7f5cf803b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.815+0000 7f5d115bf700 1 -- 172.21.15.131:0/393646634 >> v2:172.21.15.131:6800/2947584178 conn(0x7f5cf8038ef0 msgr2=0x7f5cf803b3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2947584178 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.815+0000 7f5d02ffd700 1 -- 172.21.15.131:0/393646634 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f5d080193f0 con 0x7f5d0c104d60 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.815+0000 7f5d115bf700 1 --2- 172.21.15.131:0/393646634 >> v2:172.21.15.131:6800/2947584178 conn(0x7f5cf8038ef0 0x7f5cf803b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:04.819+0000 7f5d02ffd700 1 -- 172.21.15.131:0/393646634 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5d08028940 con 0x7f5d0c104d60 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:05.016+0000 7f5d115bf700 1 -- 172.21.15.131:0/393646634 >> v2:172.21.15.131:6800/2947584178 conn(0x7f5cf8038ef0 msgr2=0x7f5cf803b3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2947584178 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:05.016+0000 7f5d115bf700 1 --2- 172.21.15.131:0/393646634 >> v2:172.21.15.131:6800/2947584178 conn(0x7f5cf8038ef0 0x7f5cf803b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:05.077+0000 7f5d14024700 1 -- 172.21.15.131:0/393646634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f5d0c063350 con 0x7f5d0c104d60 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:05.077+0000 7f5d02ffd700 1 -- 172.21.15.131:0/393646634 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f5d08028350 con 0x7f5d0c104d60 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:05.080+0000 7f5d14024700 1 -- 172.21.15.131:0/393646634 >> v2:172.21.15.131:6800/2947584178 conn(0x7f5cf8038ef0 msgr2=0x7f5cf803b3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:05.080+0000 7f5d14024700 1 --2- 172.21.15.131:0/393646634 >> v2:172.21.15.131:6800/2947584178 conn(0x7f5cf8038ef0 0x7f5cf803b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:05.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:05.080+0000 7f5d14024700 1 -- 172.21.15.131:0/393646634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d0c104d60 msgr2=0x7f5d0c19bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:05.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:05.080+0000 7f5d14024700 1 --2- 172.21.15.131:0/393646634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d0c104d60 0x7f5d0c19bdd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800d650 tx=0x7f5d080039b0 comp rx=0 tx=0).stop 2024-02-05T07:45:05.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:05.080+0000 7f5d14024700 1 -- 172.21.15.131:0/393646634 shutdown_connections 2024-02-05T07:45:05.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:05.080+0000 7f5d14024700 1 --2- 172.21.15.131:0/393646634 >> v2:172.21.15.131:6800/2947584178 conn(0x7f5cf8038ef0 0x7f5cf803b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:05.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:05.080+0000 7f5d14024700 1 --2- 172.21.15.131:0/393646634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d0c104d60 0x7f5d0c19bdd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:05.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:05.080+0000 7f5d14024700 1 -- 172.21.15.131:0/393646634 >> 172.21.15.131:0/393646634 conn(0x7f5d0c100300 msgr2=0x7f5d0c18e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:05.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:05.080+0000 7f5d14024700 1 -- 172.21.15.131:0/393646634 shutdown_connections 2024-02-05T07:45:05.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:05.080+0000 7f5d14024700 1 -- 172.21.15.131:0/393646634 wait complete. 2024-02-05T07:45:05.655 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for the mgr to restart... 2024-02-05T07:45:05.655 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr epoch 9... 2024-02-05T07:45:05.900 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/393646634' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-05T07:45:05.900 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:05 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:05.703+0000 7f9d1ca54200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T07:45:05.900 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:05 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:05.816+0000 7f9d1ca54200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T07:45:06.191 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:05 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:05.907+0000 7f9d1ca54200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T07:45:06.191 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:06 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:06.006+0000 7f9d1ca54200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T07:45:06.191 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:06 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:06.190+0000 7f9d1ca54200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T07:45:07.515 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:07 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:07.155+0000 7f9d1ca54200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T07:45:07.515 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:07 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:07.285+0000 7f9d1ca54200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T07:45:07.767 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:07 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:07.621+0000 7f9d1ca54200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T07:45:07.768 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:07 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:07.713+0000 7f9d1ca54200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T07:45:08.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:07 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:07.910+0000 7f9d1ca54200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:45:08.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:08 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:08.096+0000 7f9d1ca54200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T07:45:08.648 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:08 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:08.383+0000 7f9d1ca54200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:45:08.648 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:08 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:08.647+0000 7f9d1ca54200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T07:45:09.017 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:08 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:08.884+0000 7f9d1ca54200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T07:45:10.017 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:09 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:09.688+0000 7f9d1ca54200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T07:45:10.517 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:10 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:10.266+0000 7f9d1ca54200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T07:45:10.518 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:10 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:10.367+0000 7f9d1ca54200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T07:45:11.017 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:45:10 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:45:10.585+0000 7f9d1ca54200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T07:45:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:10 smithi131 ceph-mon[29507]: Active manager daemon y restarted 2024-02-05T07:45:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:10 smithi131 ceph-mon[29507]: Activating manager daemon y 2024-02-05T07:45:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:10 smithi131 ceph-mon[29507]: osdmap e3: 0 total, 0 up, 0 in 2024-02-05T07:45:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:10 smithi131 ceph-mon[29507]: mgrmap e10: y(active, starting, since 0.00315167s) 2024-02-05T07:45:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:45:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T07:45:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T07:45:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T07:45:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T07:45:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:10 smithi131 ceph-mon[29507]: Manager daemon y is now available 2024-02-05T07:45:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:45:11.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:45:12.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:11 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:45:12.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:11 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:45:12.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:11 smithi131 ceph-mon[29507]: mgrmap e11: y(active, since 1.0056s) 2024-02-05T07:45:12.511 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-02-05T07:45:12.511 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-05T07:45:12.512 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-05T07:45:12.512 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-02-05T07:45:12.512 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.106+0000 7f0fc7d84700 1 Processor -- start 2024-02-05T07:45:12.512 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.107+0000 7f0fc7d84700 1 -- start start 2024-02-05T07:45:12.512 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.107+0000 7f0fc7d84700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc0105e70 0x7f0fc0106290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:12.512 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.107+0000 7f0fc7d84700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc01067d0 con 0x7f0fc0105e70 2024-02-05T07:45:12.512 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.107+0000 7f0fc5b20700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc0105e70 0x7f0fc0106290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:12.512 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.107+0000 7f0fc5b20700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc0105e70 0x7f0fc0106290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34790/0 (socket says 172.21.15.131:34790) 2024-02-05T07:45:12.512 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.107+0000 7f0fc5b20700 1 -- 172.21.15.131:0/2591123934 learned_addr learned my addr 172.21.15.131:0/2591123934 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:12.512 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.107+0000 7f0fc5b20700 1 -- 172.21.15.131:0/2591123934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc0106910 con 0x7f0fc0105e70 2024-02-05T07:45:12.512 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.107+0000 7f0fc5b20700 1 --2- 172.21.15.131:0/2591123934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc0105e70 0x7f0fc0106290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0098a0 tx=0x7f0fbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd713d085522a40 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:12.512 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.107+0000 7f0fc4b1e700 1 -- 172.21.15.131:0/2591123934 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fbc004160 con 0x7f0fc0105e70 2024-02-05T07:45:12.512 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.108+0000 7f0fc4b1e700 1 -- 172.21.15.131:0/2591123934 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f0fbc0042c0 con 0x7f0fc0105e70 2024-02-05T07:45:12.512 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.108+0000 7f0fc4b1e700 1 -- 172.21.15.131:0/2591123934 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fbc011960 con 0x7f0fc0105e70 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.108+0000 7f0fc7d84700 1 -- 172.21.15.131:0/2591123934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc0105e70 msgr2=0x7f0fc0106290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.108+0000 7f0fc7d84700 1 --2- 172.21.15.131:0/2591123934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc0105e70 0x7f0fc0106290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0098a0 tx=0x7f0fbc009bb0 comp rx=0 tx=0).stop 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.108+0000 7f0fc7d84700 1 -- 172.21.15.131:0/2591123934 shutdown_connections 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.108+0000 7f0fc7d84700 1 --2- 172.21.15.131:0/2591123934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc0105e70 0x7f0fc0106290 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.108+0000 7f0fc7d84700 1 -- 172.21.15.131:0/2591123934 >> 172.21.15.131:0/2591123934 conn(0x7f0fc0101420 msgr2=0x7f0fc01038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.108+0000 7f0fc7d84700 1 -- 172.21.15.131:0/2591123934 shutdown_connections 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.108+0000 7f0fc7d84700 1 -- 172.21.15.131:0/2591123934 wait complete. 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.109+0000 7f0fc7d84700 1 Processor -- start 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.109+0000 7f0fc7d84700 1 -- start start 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.109+0000 7f0fc7d84700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc0105e70 0x7f0fc0199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.109+0000 7f0fc7d84700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc019a090 con 0x7f0fc0105e70 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.109+0000 7f0fc5b20700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc0105e70 0x7f0fc0199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.109+0000 7f0fc5b20700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc0105e70 0x7f0fc0199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34802/0 (socket says 172.21.15.131:34802) 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.109+0000 7f0fc5b20700 1 -- 172.21.15.131:0/1520621918 learned_addr learned my addr 172.21.15.131:0/1520621918 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.110+0000 7f0fc5b20700 1 -- 172.21.15.131:0/1520621918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc009550 con 0x7f0fc0105e70 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.110+0000 7f0fc5b20700 1 --2- 172.21.15.131:0/1520621918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc0105e70 0x7f0fc0199b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc000c00 tx=0x7f0fbc011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:12.513 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.110+0000 7f0fb6ffd700 1 -- 172.21.15.131:0/1520621918 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fbc011a20 con 0x7f0fc0105e70 2024-02-05T07:45:12.514 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.110+0000 7f0fc7d84700 1 -- 172.21.15.131:0/1520621918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc019a290 con 0x7f0fc0105e70 2024-02-05T07:45:12.514 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.110+0000 7f0fb6ffd700 1 -- 172.21.15.131:0/1520621918 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f0fbc003b50 con 0x7f0fc0105e70 2024-02-05T07:45:12.514 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.110+0000 7f0fb6ffd700 1 -- 172.21.15.131:0/1520621918 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fbc024da0 con 0x7f0fc0105e70 2024-02-05T07:45:12.514 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.110+0000 7f0fc7d84700 1 -- 172.21.15.131:0/1520621918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc019a730 con 0x7f0fc0105e70 2024-02-05T07:45:12.514 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.111+0000 7f0fb6ffd700 1 -- 172.21.15.131:0/1520621918 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 9) v1 ==== 45910+0+0 (secure 0 0 0) 0x7f0fbc01a480 con 0x7f0fc0105e70 2024-02-05T07:45:12.514 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.111+0000 7f0fb6ffd700 1 --2- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/2947584178 conn(0x7f0fac038ef0 0x7f0fac03b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:12.514 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.111+0000 7f0fc7d84700 1 -- 172.21.15.131:0/1520621918 --> v2:172.21.15.131:6800/2947584178 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fc0063350 con 0x7f0fac038ef0 2024-02-05T07:45:12.514 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.111+0000 7f0fb6ffd700 1 -- 172.21.15.131:0/1520621918 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f0fbc04d8e0 con 0x7f0fc0105e70 2024-02-05T07:45:12.514 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.111+0000 7f0fc531f700 1 -- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/2947584178 conn(0x7f0fac038ef0 msgr2=0x7f0fac03b3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2947584178 2024-02-05T07:45:12.514 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.111+0000 7f0fc531f700 1 --2- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/2947584178 conn(0x7f0fac038ef0 0x7f0fac03b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-05T07:45:12.514 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.312+0000 7f0fc531f700 1 -- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/2947584178 conn(0x7f0fac038ef0 msgr2=0x7f0fac03b3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2947584178 2024-02-05T07:45:12.514 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.312+0000 7f0fc531f700 1 --2- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/2947584178 conn(0x7f0fac038ef0 0x7f0fac03b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-05T07:45:12.514 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.713+0000 7f0fc531f700 1 -- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/2947584178 conn(0x7f0fac038ef0 msgr2=0x7f0fac03b3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2947584178 2024-02-05T07:45:12.514 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:06.713+0000 7f0fc531f700 1 --2- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/2947584178 conn(0x7f0fac038ef0 0x7f0fac03b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-05T07:45:12.514 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:07.514+0000 7f0fc531f700 1 -- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/2947584178 conn(0x7f0fac038ef0 msgr2=0x7f0fac03b3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2947584178 2024-02-05T07:45:12.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:07.514+0000 7f0fc531f700 1 --2- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/2947584178 conn(0x7f0fac038ef0 0x7f0fac03b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-05T07:45:12.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:09.115+0000 7f0fc531f700 1 -- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/2947584178 conn(0x7f0fac038ef0 msgr2=0x7f0fac03b3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/2947584178 2024-02-05T07:45:12.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:09.115+0000 7f0fc531f700 1 --2- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/2947584178 conn(0x7f0fac038ef0 0x7f0fac03b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-05T07:45:12.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:10.600+0000 7f0fb6ffd700 1 -- 172.21.15.131:0/1520621918 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mgrmap(e 10) v1 ==== 45712+0+0 (secure 0 0 0) 0x7f0fbc01a910 con 0x7f0fc0105e70 2024-02-05T07:45:12.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:10.600+0000 7f0fb6ffd700 1 -- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/2947584178 conn(0x7f0fac038ef0 msgr2=0x7f0fac03b3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-05T07:45:12.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:10.600+0000 7f0fb6ffd700 1 --2- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/2947584178 conn(0x7f0fac038ef0 0x7f0fac03b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:12.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.601+0000 7f0fb6ffd700 1 -- 172.21.15.131:0/1520621918 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 11) v1 ==== 45804+0+0 (secure 0 0 0) 0x7f0fbc04e4d0 con 0x7f0fc0105e70 2024-02-05T07:45:12.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.601+0000 7f0fb6ffd700 1 --2- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/760045032 conn(0x7f0fac03c720 0x7f0fac03eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:12.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.601+0000 7f0fb6ffd700 1 -- 172.21.15.131:0/1520621918 --> v2:172.21.15.131:6800/760045032 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fc0063350 con 0x7f0fac03c720 2024-02-05T07:45:12.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.602+0000 7f0fc531f700 1 --2- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/760045032 conn(0x7f0fac03c720 0x7f0fac03eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:12.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.603+0000 7f0fc531f700 1 --2- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/760045032 conn(0x7f0fac03c720 0x7f0fac03eb10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0003a10 tx=0x7f0fb00092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:12.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.603+0000 7f0fb6ffd700 1 -- 172.21.15.131:0/1520621918 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f0fc0063350 con 0x7f0fac03c720 2024-02-05T07:45:12.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.616+0000 7f0fc7d84700 1 -- 172.21.15.131:0/1520621918 --> v2:172.21.15.131:6800/760045032 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f0fc0107d20 con 0x7f0fac03c720 2024-02-05T07:45:12.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.616+0000 7f0fb6ffd700 1 -- 172.21.15.131:0/1520621918 <== mgr.14152 v2:172.21.15.131:6800/760045032 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f0fc0107d20 con 0x7f0fac03c720 2024-02-05T07:45:12.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.616+0000 7f0fc7d84700 1 -- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/760045032 conn(0x7f0fac03c720 msgr2=0x7f0fac03eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:12.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.616+0000 7f0fc7d84700 1 --2- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/760045032 conn(0x7f0fac03c720 0x7f0fac03eb10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0003a10 tx=0x7f0fb00092b0 comp rx=0 tx=0).stop 2024-02-05T07:45:12.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.616+0000 7f0fc7d84700 1 -- 172.21.15.131:0/1520621918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc0105e70 msgr2=0x7f0fc0199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:12.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.616+0000 7f0fc7d84700 1 --2- 172.21.15.131:0/1520621918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc0105e70 0x7f0fc0199b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc000c00 tx=0x7f0fbc011eb0 comp rx=0 tx=0).stop 2024-02-05T07:45:12.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.616+0000 7f0fc7d84700 1 -- 172.21.15.131:0/1520621918 shutdown_connections 2024-02-05T07:45:12.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.616+0000 7f0fc7d84700 1 --2- 172.21.15.131:0/1520621918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fc0105e70 0x7f0fc0199b50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:12.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.616+0000 7f0fc7d84700 1 --2- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/760045032 conn(0x7f0fac03c720 0x7f0fac03eb10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:12.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.616+0000 7f0fc7d84700 1 --2- 172.21.15.131:0/1520621918 >> v2:172.21.15.131:6800/2947584178 conn(0x7f0fac038ef0 0x7f0fac03b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:12.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.616+0000 7f0fc7d84700 1 -- 172.21.15.131:0/1520621918 >> 172.21.15.131:0/1520621918 conn(0x7f0fc0101420 msgr2=0x7f0fc0102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:12.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.617+0000 7f0fc7d84700 1 -- 172.21.15.131:0/1520621918 shutdown_connections 2024-02-05T07:45:12.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:11.617+0000 7f0fc7d84700 1 -- 172.21.15.131:0/1520621918 wait complete. 2024-02-05T07:45:12.516 INFO:teuthology.orchestra.run.smithi131.stdout:mgr epoch 9 is available 2024-02-05T07:45:12.516 INFO:teuthology.orchestra.run.smithi131.stdout:Generating a dashboard self-signed certificate... 2024-02-05T07:45:12.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:12 smithi131 ceph-mon[29507]: [05/Feb/2024:07:45:11] ENGINE Bus STARTING 2024-02-05T07:45:12.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:12 smithi131 ceph-mon[29507]: [05/Feb/2024:07:45:11] ENGINE Serving on http://172.21.15.131:8765 2024-02-05T07:45:12.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:12 smithi131 ceph-mon[29507]: [05/Feb/2024:07:45:11] ENGINE Serving on https://172.21.15.131:7150 2024-02-05T07:45:12.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:12 smithi131 ceph-mon[29507]: [05/Feb/2024:07:45:11] ENGINE Bus STARTED 2024-02-05T07:45:12.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:12 smithi131 ceph-mon[29507]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-05T07:45:12.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:12 smithi131 ceph-mon[29507]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-05T07:45:12.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:12 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:13.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-05T07:45:13.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.984+0000 7f130769c700 1 Processor -- start 2024-02-05T07:45:13.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.984+0000 7f130769c700 1 -- start start 2024-02-05T07:45:13.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.984+0000 7f130769c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1300105e60 0x7f1300106280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:13.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.984+0000 7f130769c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13001067c0 con 0x7f1300105e60 2024-02-05T07:45:13.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.984+0000 7f1305438700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1300105e60 0x7f1300106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:13.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.984+0000 7f1305438700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1300105e60 0x7f1300106280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54628/0 (socket says 172.21.15.131:54628) 2024-02-05T07:45:13.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.984+0000 7f1305438700 1 -- 172.21.15.131:0/1398323151 learned_addr learned my addr 172.21.15.131:0/1398323151 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.985+0000 7f1305438700 1 -- 172.21.15.131:0/1398323151 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1300106900 con 0x7f1300105e60 2024-02-05T07:45:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.985+0000 7f1305438700 1 --2- 172.21.15.131:0/1398323151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1300105e60 0x7f1300106280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0098a0 tx=0x7f12fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ec5958600ce0c7 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.985+0000 7f12f7fff700 1 -- 172.21.15.131:0/1398323151 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12fc004160 con 0x7f1300105e60 2024-02-05T07:45:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.985+0000 7f12f7fff700 1 -- 172.21.15.131:0/1398323151 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f12fc0042c0 con 0x7f1300105e60 2024-02-05T07:45:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.985+0000 7f12f7fff700 1 -- 172.21.15.131:0/1398323151 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12fc011960 con 0x7f1300105e60 2024-02-05T07:45:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.986+0000 7f130769c700 1 -- 172.21.15.131:0/1398323151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1300105e60 msgr2=0x7f1300106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.986+0000 7f130769c700 1 --2- 172.21.15.131:0/1398323151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1300105e60 0x7f1300106280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0098a0 tx=0x7f12fc009bb0 comp rx=0 tx=0).stop 2024-02-05T07:45:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.986+0000 7f130769c700 1 -- 172.21.15.131:0/1398323151 shutdown_connections 2024-02-05T07:45:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.986+0000 7f130769c700 1 --2- 172.21.15.131:0/1398323151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1300105e60 0x7f1300106280 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.986+0000 7f130769c700 1 -- 172.21.15.131:0/1398323151 >> 172.21.15.131:0/1398323151 conn(0x7f13001014d0 msgr2=0x7f13001038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.986+0000 7f130769c700 1 -- 172.21.15.131:0/1398323151 shutdown_connections 2024-02-05T07:45:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.986+0000 7f130769c700 1 -- 172.21.15.131:0/1398323151 wait complete. 2024-02-05T07:45:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.986+0000 7f130769c700 1 Processor -- start 2024-02-05T07:45:13.942 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.986+0000 7f130769c700 1 -- start start 2024-02-05T07:45:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.986+0000 7f130769c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1300105e60 0x7f1300199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.986+0000 7f130769c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130019a0e0 con 0x7f1300105e60 2024-02-05T07:45:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.987+0000 7f1305438700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1300105e60 0x7f1300199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.987+0000 7f1305438700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1300105e60 0x7f1300199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54636/0 (socket says 172.21.15.131:54636) 2024-02-05T07:45:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.987+0000 7f1305438700 1 -- 172.21.15.131:0/2622635697 learned_addr learned my addr 172.21.15.131:0/2622635697 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.987+0000 7f1305438700 1 -- 172.21.15.131:0/2622635697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc009550 con 0x7f1300105e60 2024-02-05T07:45:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.987+0000 7f1305438700 1 --2- 172.21.15.131:0/2622635697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1300105e60 0x7f1300199ba0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f12fc006af0 tx=0x7f12fc011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.987+0000 7f12f67fc700 1 -- 172.21.15.131:0/2622635697 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12fc0036a0 con 0x7f1300105e60 2024-02-05T07:45:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.987+0000 7f12f67fc700 1 -- 172.21.15.131:0/2622635697 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f12fc01a430 con 0x7f1300105e60 2024-02-05T07:45:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.987+0000 7f130769c700 1 -- 172.21.15.131:0/2622635697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130019a2e0 con 0x7f1300105e60 2024-02-05T07:45:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.987+0000 7f130769c700 1 -- 172.21.15.131:0/2622635697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130019a780 con 0x7f1300105e60 2024-02-05T07:45:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.987+0000 7f12f67fc700 1 -- 172.21.15.131:0/2622635697 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12fc0193c0 con 0x7f1300105e60 2024-02-05T07:45:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.988+0000 7f12f67fc700 1 -- 172.21.15.131:0/2622635697 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 11) v1 ==== 45804+0+0 (secure 0 0 0) 0x7f12fc019640 con 0x7f1300105e60 2024-02-05T07:45:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.988+0000 7f130769c700 1 -- 172.21.15.131:0/2622635697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e4005150 con 0x7f1300105e60 2024-02-05T07:45:13.943 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.988+0000 7f12f67fc700 1 --2- 172.21.15.131:0/2622635697 >> v2:172.21.15.131:6800/760045032 conn(0x7f12ec038dd0 0x7f12ec03b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.988+0000 7f12f67fc700 1 -- 172.21.15.131:0/2622635697 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f12fc04cd50 con 0x7f1300105e60 2024-02-05T07:45:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.989+0000 7f1304c37700 1 --2- 172.21.15.131:0/2622635697 >> v2:172.21.15.131:6800/760045032 conn(0x7f12ec038dd0 0x7f12ec03b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.989+0000 7f1304c37700 1 --2- 172.21.15.131:0/2622635697 >> v2:172.21.15.131:6800/760045032 conn(0x7f12ec038dd0 0x7f12ec03b290 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f12f0006fd0 tx=0x7f12f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:12.993+0000 7f12f67fc700 1 -- 172.21.15.131:0/2622635697 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f12fc019e60 con 0x7f1300105e60 2024-02-05T07:45:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:13.167+0000 7f12f67fc700 1 -- 172.21.15.131:0/2622635697 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 12) v1 ==== 45910+0+0 (secure 0 0 0) 0x7f12fc052020 con 0x7f1300105e60 2024-02-05T07:45:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:13.195+0000 7f130769c700 1 -- 172.21.15.131:0/2622635697 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f12e4000bf0 con 0x7f12ec038dd0 2024-02-05T07:45:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:13.343+0000 7f12f67fc700 1 -- 172.21.15.131:0/2622635697 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f12e4000bf0 con 0x7f12ec038dd0 2024-02-05T07:45:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:13.346+0000 7f130769c700 1 -- 172.21.15.131:0/2622635697 >> v2:172.21.15.131:6800/760045032 conn(0x7f12ec038dd0 msgr2=0x7f12ec03b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:13.346+0000 7f130769c700 1 --2- 172.21.15.131:0/2622635697 >> v2:172.21.15.131:6800/760045032 conn(0x7f12ec038dd0 0x7f12ec03b290 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f12f0006fd0 tx=0x7f12f0006e40 comp rx=0 tx=0).stop 2024-02-05T07:45:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:13.346+0000 7f130769c700 1 -- 172.21.15.131:0/2622635697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1300105e60 msgr2=0x7f1300199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:13.346+0000 7f130769c700 1 --2- 172.21.15.131:0/2622635697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1300105e60 0x7f1300199ba0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f12fc006af0 tx=0x7f12fc011eb0 comp rx=0 tx=0).stop 2024-02-05T07:45:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:13.346+0000 7f130769c700 1 -- 172.21.15.131:0/2622635697 shutdown_connections 2024-02-05T07:45:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:13.346+0000 7f130769c700 1 --2- 172.21.15.131:0/2622635697 >> v2:172.21.15.131:6800/760045032 conn(0x7f12ec038dd0 0x7f12ec03b290 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:13.346+0000 7f130769c700 1 --2- 172.21.15.131:0/2622635697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1300105e60 0x7f1300199ba0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:13.944 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:13.346+0000 7f130769c700 1 -- 172.21.15.131:0/2622635697 >> 172.21.15.131:0/2622635697 conn(0x7f13001014d0 msgr2=0x7f13001020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:13.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:13.346+0000 7f130769c700 1 -- 172.21.15.131:0/2622635697 shutdown_connections 2024-02-05T07:45:13.945 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:13.346+0000 7f130769c700 1 -- 172.21.15.131:0/2622635697 wait complete. 2024-02-05T07:45:13.945 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial admin user... 2024-02-05T07:45:14.187 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:14 smithi131 ceph-mon[29507]: mgrmap e12: y(active, since 2s) 2024-02-05T07:45:14.187 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:14 smithi131 ceph-mon[29507]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:45:14.187 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:14 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:14.187 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:14 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:14.187 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:14 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:15.368 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:15 smithi131 ceph-mon[29507]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:45:15.368 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:15 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:15.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$jeHNpUJ0bJVuGe6iX/ay9uppYZPZlg3Qw3rj8bZ3tuu7zTlXZ18Ou", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707119114, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-05T07:45:15.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.418+0000 7f0c4eec9700 1 Processor -- start 2024-02-05T07:45:15.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.418+0000 7f0c4eec9700 1 -- start start 2024-02-05T07:45:15.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.418+0000 7f0c4eec9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c48106930 0x7f0c48108d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:15.653 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.418+0000 7f0c4eec9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48078df0 con 0x7f0c48106930 2024-02-05T07:45:15.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.418+0000 7f0c4cc65700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c48106930 0x7f0c48108d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:15.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.418+0000 7f0c4cc65700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c48106930 0x7f0c48108d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54640/0 (socket says 172.21.15.131:54640) 2024-02-05T07:45:15.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.418+0000 7f0c4cc65700 1 -- 172.21.15.131:0/278747810 learned_addr learned my addr 172.21.15.131:0/278747810 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:15.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.419+0000 7f0c4cc65700 1 -- 172.21.15.131:0/278747810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c48078f30 con 0x7f0c48106930 2024-02-05T07:45:15.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.419+0000 7f0c4cc65700 1 --2- 172.21.15.131:0/278747810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c48106930 0x7f0c48108d20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0c40009e80 tx=0x7f0c4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d0d6d2844e3176 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:15.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.419+0000 7f0c477fe700 1 -- 172.21.15.131:0/278747810 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c4000e070 con 0x7f0c48106930 2024-02-05T07:45:15.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.419+0000 7f0c477fe700 1 -- 172.21.15.131:0/278747810 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f0c4000b8c0 con 0x7f0c48106930 2024-02-05T07:45:15.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.419+0000 7f0c477fe700 1 -- 172.21.15.131:0/278747810 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c400043d0 con 0x7f0c48106930 2024-02-05T07:45:15.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.420+0000 7f0c4eec9700 1 -- 172.21.15.131:0/278747810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c48106930 msgr2=0x7f0c48108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:15.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.420+0000 7f0c4eec9700 1 --2- 172.21.15.131:0/278747810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c48106930 0x7f0c48108d20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0c40009e80 tx=0x7f0c4000b330 comp rx=0 tx=0).stop 2024-02-05T07:45:15.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.420+0000 7f0c4eec9700 1 -- 172.21.15.131:0/278747810 shutdown_connections 2024-02-05T07:45:15.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.420+0000 7f0c4eec9700 1 --2- 172.21.15.131:0/278747810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c48106930 0x7f0c48108d20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:15.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.420+0000 7f0c4eec9700 1 -- 172.21.15.131:0/278747810 >> 172.21.15.131:0/278747810 conn(0x7f0c481002c0 msgr2=0x7f0c481026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:15.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.420+0000 7f0c4eec9700 1 -- 172.21.15.131:0/278747810 shutdown_connections 2024-02-05T07:45:15.654 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.420+0000 7f0c4eec9700 1 -- 172.21.15.131:0/278747810 wait complete. 2024-02-05T07:45:15.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.420+0000 7f0c4eec9700 1 Processor -- start 2024-02-05T07:45:15.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.421+0000 7f0c4eec9700 1 -- start start 2024-02-05T07:45:15.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.421+0000 7f0c4eec9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c48106930 0x7f0c48199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:15.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.421+0000 7f0c4eec9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48199f50 con 0x7f0c48106930 2024-02-05T07:45:15.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.421+0000 7f0c4cc65700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c48106930 0x7f0c48199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:15.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.421+0000 7f0c4cc65700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c48106930 0x7f0c48199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54644/0 (socket says 172.21.15.131:54644) 2024-02-05T07:45:15.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.421+0000 7f0c4cc65700 1 -- 172.21.15.131:0/17573495 learned_addr learned my addr 172.21.15.131:0/17573495 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:15.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.421+0000 7f0c4cc65700 1 -- 172.21.15.131:0/17573495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c40009b30 con 0x7f0c48106930 2024-02-05T07:45:15.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.421+0000 7f0c4cc65700 1 --2- 172.21.15.131:0/17573495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c48106930 0x7f0c48199a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c400072a0 tx=0x7f0c40003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:15.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.421+0000 7f0c45ffb700 1 -- 172.21.15.131:0/17573495 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c4000e040 con 0x7f0c48106930 2024-02-05T07:45:15.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.421+0000 7f0c45ffb700 1 -- 172.21.15.131:0/17573495 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f0c4000ba30 con 0x7f0c48106930 2024-02-05T07:45:15.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.421+0000 7f0c4eec9700 1 -- 172.21.15.131:0/17573495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4819a150 con 0x7f0c48106930 2024-02-05T07:45:15.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.422+0000 7f0c45ffb700 1 -- 172.21.15.131:0/17573495 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c400125a0 con 0x7f0c48106930 2024-02-05T07:45:15.655 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.422+0000 7f0c4eec9700 1 -- 172.21.15.131:0/17573495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4819a650 con 0x7f0c48106930 2024-02-05T07:45:15.656 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.422+0000 7f0c45ffb700 1 -- 172.21.15.131:0/17573495 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 12) v1 ==== 45910+0+0 (secure 0 0 0) 0x7f0c40012820 con 0x7f0c48106930 2024-02-05T07:45:15.656 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.423+0000 7f0c45ffb700 1 --2- 172.21.15.131:0/17573495 >> v2:172.21.15.131:6800/760045032 conn(0x7f0c30038ea0 0x7f0c3003b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:15.656 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.423+0000 7f0c45ffb700 1 -- 172.21.15.131:0/17573495 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f0c4004d190 con 0x7f0c48106930 2024-02-05T07:45:15.656 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.423+0000 7f0c4eec9700 1 -- 172.21.15.131:0/17573495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c48109fd0 con 0x7f0c48106930 2024-02-05T07:45:15.656 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.423+0000 7f0c47fff700 1 --2- 172.21.15.131:0/17573495 >> v2:172.21.15.131:6800/760045032 conn(0x7f0c30038ea0 0x7f0c3003b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:15.656 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.423+0000 7f0c47fff700 1 --2- 172.21.15.131:0/17573495 >> v2:172.21.15.131:6800/760045032 conn(0x7f0c30038ea0 0x7f0c3003b360 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c006fd0 tx=0x7f0c3c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:15.656 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.428+0000 7f0c45ffb700 1 -- 172.21.15.131:0/17573495 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0c40020080 con 0x7f0c48106930 2024-02-05T07:45:15.656 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.646+0000 7f0c4eec9700 1 -- 172.21.15.131:0/17573495 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0c4810a2e0 con 0x7f0c30038ea0 2024-02-05T07:45:15.656 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.871+0000 7f0c45ffb700 1 -- 172.21.15.131:0/17573495 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f0c4810a2e0 con 0x7f0c30038ea0 2024-02-05T07:45:15.656 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.873+0000 7f0c4eec9700 1 -- 172.21.15.131:0/17573495 >> v2:172.21.15.131:6800/760045032 conn(0x7f0c30038ea0 msgr2=0x7f0c3003b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:15.656 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.873+0000 7f0c4eec9700 1 --2- 172.21.15.131:0/17573495 >> v2:172.21.15.131:6800/760045032 conn(0x7f0c30038ea0 0x7f0c3003b360 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c006fd0 tx=0x7f0c3c006e40 comp rx=0 tx=0).stop 2024-02-05T07:45:15.656 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.873+0000 7f0c4eec9700 1 -- 172.21.15.131:0/17573495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c48106930 msgr2=0x7f0c48199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:15.656 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.873+0000 7f0c4eec9700 1 --2- 172.21.15.131:0/17573495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c48106930 0x7f0c48199a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c400072a0 tx=0x7f0c40003d00 comp rx=0 tx=0).stop 2024-02-05T07:45:15.656 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.873+0000 7f0c4eec9700 1 -- 172.21.15.131:0/17573495 shutdown_connections 2024-02-05T07:45:15.656 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.873+0000 7f0c4eec9700 1 --2- 172.21.15.131:0/17573495 >> v2:172.21.15.131:6800/760045032 conn(0x7f0c30038ea0 0x7f0c3003b360 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:15.657 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.873+0000 7f0c4eec9700 1 --2- 172.21.15.131:0/17573495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c48106930 0x7f0c48199a10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:15.657 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.873+0000 7f0c4eec9700 1 -- 172.21.15.131:0/17573495 >> 172.21.15.131:0/17573495 conn(0x7f0c481002c0 msgr2=0x7f0c48100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:15.657 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.873+0000 7f0c4eec9700 1 -- 172.21.15.131:0/17573495 shutdown_connections 2024-02-05T07:45:15.657 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:14.873+0000 7f0c4eec9700 1 -- 172.21.15.131:0/17573495 wait complete. 2024-02-05T07:45:15.657 INFO:teuthology.orchestra.run.smithi131.stdout:Fetching dashboard port number... 2024-02-05T07:45:17.463 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 8443 2024-02-05T07:45:17.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.621+0000 7f1a852b9700 1 Processor -- start 2024-02-05T07:45:17.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.621+0000 7f1a852b9700 1 -- start start 2024-02-05T07:45:17.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.621+0000 7f1a852b9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a80105e70 0x7f1a80106290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:17.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.621+0000 7f1a852b9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a801067d0 con 0x7f1a80105e70 2024-02-05T07:45:17.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.622+0000 7f1a7effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a80105e70 0x7f1a80106290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:17.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.622+0000 7f1a7effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a80105e70 0x7f1a80106290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54648/0 (socket says 172.21.15.131:54648) 2024-02-05T07:45:17.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.622+0000 7f1a7effd700 1 -- 172.21.15.131:0/1292393757 learned_addr learned my addr 172.21.15.131:0/1292393757 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:17.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.622+0000 7f1a7effd700 1 -- 172.21.15.131:0/1292393757 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a80106910 con 0x7f1a80105e70 2024-02-05T07:45:17.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.622+0000 7f1a7effd700 1 --2- 172.21.15.131:0/1292393757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a80105e70 0x7f1a80106290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1a700098a0 tx=0x7f1a70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a490fa0dc1f3a56 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:17.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.622+0000 7f1a7e7fc700 1 -- 172.21.15.131:0/1292393757 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a70004160 con 0x7f1a80105e70 2024-02-05T07:45:17.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.622+0000 7f1a7e7fc700 1 -- 172.21.15.131:0/1292393757 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f1a700042c0 con 0x7f1a80105e70 2024-02-05T07:45:17.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.622+0000 7f1a7e7fc700 1 -- 172.21.15.131:0/1292393757 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a70011960 con 0x7f1a80105e70 2024-02-05T07:45:17.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.623+0000 7f1a852b9700 1 -- 172.21.15.131:0/1292393757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a80105e70 msgr2=0x7f1a80106290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:17.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.623+0000 7f1a852b9700 1 --2- 172.21.15.131:0/1292393757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a80105e70 0x7f1a80106290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1a700098a0 tx=0x7f1a70009bb0 comp rx=0 tx=0).stop 2024-02-05T07:45:17.464 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.623+0000 7f1a852b9700 1 -- 172.21.15.131:0/1292393757 shutdown_connections 2024-02-05T07:45:17.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.623+0000 7f1a852b9700 1 --2- 172.21.15.131:0/1292393757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a80105e70 0x7f1a80106290 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:17.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.623+0000 7f1a852b9700 1 -- 172.21.15.131:0/1292393757 >> 172.21.15.131:0/1292393757 conn(0x7f1a80101420 msgr2=0x7f1a801038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:17.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.623+0000 7f1a852b9700 1 -- 172.21.15.131:0/1292393757 shutdown_connections 2024-02-05T07:45:17.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.623+0000 7f1a852b9700 1 -- 172.21.15.131:0/1292393757 wait complete. 2024-02-05T07:45:17.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.624+0000 7f1a852b9700 1 Processor -- start 2024-02-05T07:45:17.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.624+0000 7f1a852b9700 1 -- start start 2024-02-05T07:45:17.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.624+0000 7f1a852b9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a80105e70 0x7f1a80199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:17.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.624+0000 7f1a852b9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8019a090 con 0x7f1a80105e70 2024-02-05T07:45:17.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.624+0000 7f1a7effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a80105e70 0x7f1a80199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:17.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.624+0000 7f1a7effd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a80105e70 0x7f1a80199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54652/0 (socket says 172.21.15.131:54652) 2024-02-05T07:45:17.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.624+0000 7f1a7effd700 1 -- 172.21.15.131:0/1823591435 learned_addr learned my addr 172.21.15.131:0/1823591435 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:17.465 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.624+0000 7f1a7effd700 1 -- 172.21.15.131:0/1823591435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a70009550 con 0x7f1a80105e70 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.624+0000 7f1a7effd700 1 --2- 172.21.15.131:0/1823591435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a80105e70 0x7f1a80199b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009fb0 tx=0x7f1a700119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.624+0000 7f1a7cff9700 1 -- 172.21.15.131:0/1823591435 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a70003970 con 0x7f1a80105e70 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.624+0000 7f1a7cff9700 1 -- 172.21.15.131:0/1823591435 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f1a70003ad0 con 0x7f1a80105e70 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.624+0000 7f1a852b9700 1 -- 172.21.15.131:0/1823591435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8019a290 con 0x7f1a80105e70 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.624+0000 7f1a7cff9700 1 -- 172.21.15.131:0/1823591435 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a70019630 con 0x7f1a80105e70 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.624+0000 7f1a852b9700 1 -- 172.21.15.131:0/1823591435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8019a730 con 0x7f1a80105e70 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.625+0000 7f1a7cff9700 1 -- 172.21.15.131:0/1823591435 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 12) v1 ==== 45910+0+0 (secure 0 0 0) 0x7f1a7001a430 con 0x7f1a80105e70 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.625+0000 7f1a7cff9700 1 --2- 172.21.15.131:0/1823591435 >> v2:172.21.15.131:6800/760045032 conn(0x7f1a6c038ea0 0x7f1a6c03b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.625+0000 7f1a852b9700 1 -- 172.21.15.131:0/1823591435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a80063350 con 0x7f1a80105e70 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.626+0000 7f1a7cff9700 1 -- 172.21.15.131:0/1823591435 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f1a7004d3d0 con 0x7f1a80105e70 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.626+0000 7f1a77fff700 1 --2- 172.21.15.131:0/1823591435 >> v2:172.21.15.131:6800/760045032 conn(0x7f1a6c038ea0 0x7f1a6c03b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.626+0000 7f1a77fff700 1 --2- 172.21.15.131:0/1823591435 >> v2:172.21.15.131:6800/760045032 conn(0x7f1a6c038ea0 0x7f1a6c03b360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1a68006fd0 tx=0x7f1a68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.630+0000 7f1a7cff9700 1 -- 172.21.15.131:0/1823591435 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1a70018530 con 0x7f1a80105e70 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.687+0000 7f1a7cff9700 1 -- 172.21.15.131:0/1823591435 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7f1a7000fda0 con 0x7f1a80105e70 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.823+0000 7f1a852b9700 1 -- 172.21.15.131:0/1823591435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f1a8019d120 con 0x7f1a80105e70 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.823+0000 7f1a7cff9700 1 -- 172.21.15.131:0/1823591435 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f1a70019e10 con 0x7f1a80105e70 2024-02-05T07:45:17.466 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.826+0000 7f1a852b9700 1 -- 172.21.15.131:0/1823591435 >> v2:172.21.15.131:6800/760045032 conn(0x7f1a6c038ea0 msgr2=0x7f1a6c03b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:17.467 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.826+0000 7f1a852b9700 1 --2- 172.21.15.131:0/1823591435 >> v2:172.21.15.131:6800/760045032 conn(0x7f1a6c038ea0 0x7f1a6c03b360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1a68006fd0 tx=0x7f1a68006e40 comp rx=0 tx=0).stop 2024-02-05T07:45:17.467 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.826+0000 7f1a852b9700 1 -- 172.21.15.131:0/1823591435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a80105e70 msgr2=0x7f1a80199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:17.467 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.826+0000 7f1a852b9700 1 --2- 172.21.15.131:0/1823591435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a80105e70 0x7f1a80199b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009fb0 tx=0x7f1a700119e0 comp rx=0 tx=0).stop 2024-02-05T07:45:17.467 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.826+0000 7f1a852b9700 1 -- 172.21.15.131:0/1823591435 shutdown_connections 2024-02-05T07:45:17.467 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.826+0000 7f1a852b9700 1 --2- 172.21.15.131:0/1823591435 >> v2:172.21.15.131:6800/760045032 conn(0x7f1a6c038ea0 0x7f1a6c03b360 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:17.467 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.826+0000 7f1a852b9700 1 --2- 172.21.15.131:0/1823591435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a80105e70 0x7f1a80199b50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:17.467 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.826+0000 7f1a852b9700 1 -- 172.21.15.131:0/1823591435 >> 172.21.15.131:0/1823591435 conn(0x7f1a80101420 msgr2=0x7f1a80102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:17.467 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.826+0000 7f1a852b9700 1 -- 172.21.15.131:0/1823591435 shutdown_connections 2024-02-05T07:45:17.467 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:16.826+0000 7f1a852b9700 1 -- 172.21.15.131:0/1823591435 wait complete. 2024-02-05T07:45:17.469 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-05T07:45:17.470 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-02-05T07:45:17.476 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-05T07:45:17.476 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-02-05T07:45:17.477 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-02-05T07:45:17.477 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-05T07:45:17.478 INFO:teuthology.orchestra.run.smithi131.stdout:Ceph Dashboard is now available at: 2024-02-05T07:45:17.478 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:45:17.479 INFO:teuthology.orchestra.run.smithi131.stdout: URL: https://smithi131.front.sepia.ceph.com:8443/ 2024-02-05T07:45:17.479 INFO:teuthology.orchestra.run.smithi131.stdout: User: admin 2024-02-05T07:45:17.479 INFO:teuthology.orchestra.run.smithi131.stdout: Password: 6ycg686w39 2024-02-05T07:45:17.479 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:45:17.479 INFO:teuthology.orchestra.run.smithi131.stdout:Saving cluster configuration to /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config directory 2024-02-05T07:45:17.479 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling autotune for osd_memory_target 2024-02-05T07:45:17.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:17 smithi131 ceph-mon[29507]: mgrmap e13: y(active, since 6s) 2024-02-05T07:45:17.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1823591435' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-05T07:45:18.698 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.905+0000 7f8970398700 1 Processor -- start 2024-02-05T07:45:18.703 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.905+0000 7f8970398700 1 -- start start 2024-02-05T07:45:18.703 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.905+0000 7f8970398700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89681040f0 0x7f8968104510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:18.703 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.905+0000 7f8970398700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8968104ae0 con 0x7f89681040f0 2024-02-05T07:45:18.703 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.905+0000 7f896e134700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89681040f0 0x7f8968104510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:18.703 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.905+0000 7f896e134700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89681040f0 0x7f8968104510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54662/0 (socket says 172.21.15.131:54662) 2024-02-05T07:45:18.703 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.905+0000 7f896e134700 1 -- 172.21.15.131:0/2471034994 learned_addr learned my addr 172.21.15.131:0/2471034994 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:18.703 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.905+0000 7f896e134700 1 -- 172.21.15.131:0/2471034994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8968104c20 con 0x7f89681040f0 2024-02-05T07:45:18.704 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.906+0000 7f896e134700 1 --2- 172.21.15.131:0/2471034994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89681040f0 0x7f8968104510 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f89580098a0 tx=0x7f8958009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3624fc3d13c4db78 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:18.704 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.906+0000 7f896d132700 1 -- 172.21.15.131:0/2471034994 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8958004160 con 0x7f89681040f0 2024-02-05T07:45:18.704 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.906+0000 7f896d132700 1 -- 172.21.15.131:0/2471034994 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f89580042c0 con 0x7f89681040f0 2024-02-05T07:45:18.704 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.906+0000 7f896d132700 1 -- 172.21.15.131:0/2471034994 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8958011960 con 0x7f89681040f0 2024-02-05T07:45:18.704 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.906+0000 7f8970398700 1 -- 172.21.15.131:0/2471034994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89681040f0 msgr2=0x7f8968104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:18.704 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.906+0000 7f8970398700 1 --2- 172.21.15.131:0/2471034994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89681040f0 0x7f8968104510 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f89580098a0 tx=0x7f8958009bb0 comp rx=0 tx=0).stop 2024-02-05T07:45:18.704 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.907+0000 7f8970398700 1 -- 172.21.15.131:0/2471034994 shutdown_connections 2024-02-05T07:45:18.704 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.907+0000 7f8970398700 1 --2- 172.21.15.131:0/2471034994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89681040f0 0x7f8968104510 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:18.704 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.907+0000 7f8970398700 1 -- 172.21.15.131:0/2471034994 >> 172.21.15.131:0/2471034994 conn(0x7f89680ffa40 msgr2=0x7f8968101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:18.704 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.907+0000 7f8970398700 1 -- 172.21.15.131:0/2471034994 shutdown_connections 2024-02-05T07:45:18.704 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.907+0000 7f8970398700 1 -- 172.21.15.131:0/2471034994 wait complete. 2024-02-05T07:45:18.704 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.907+0000 7f8970398700 1 Processor -- start 2024-02-05T07:45:18.704 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.907+0000 7f8970398700 1 -- start start 2024-02-05T07:45:18.704 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.907+0000 7f8970398700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89681040f0 0x7f8968199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:18.705 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.907+0000 7f8970398700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896819a040 con 0x7f89681040f0 2024-02-05T07:45:18.705 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.908+0000 7f896e134700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89681040f0 0x7f8968199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:18.705 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.908+0000 7f896e134700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89681040f0 0x7f8968199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54674/0 (socket says 172.21.15.131:54674) 2024-02-05T07:45:18.705 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.908+0000 7f896e134700 1 -- 172.21.15.131:0/1089493072 learned_addr learned my addr 172.21.15.131:0/1089493072 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:18.705 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.908+0000 7f896e134700 1 -- 172.21.15.131:0/1089493072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8958009550 con 0x7f89681040f0 2024-02-05T07:45:18.705 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.908+0000 7f896e134700 1 --2- 172.21.15.131:0/1089493072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89681040f0 0x7f8968199b00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8958006af0 tx=0x7f89580119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:18.705 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.908+0000 7f895f7fe700 1 -- 172.21.15.131:0/1089493072 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89580038b0 con 0x7f89681040f0 2024-02-05T07:45:18.705 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.908+0000 7f8970398700 1 -- 172.21.15.131:0/1089493072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f896819a240 con 0x7f89681040f0 2024-02-05T07:45:18.705 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.908+0000 7f895f7fe700 1 -- 172.21.15.131:0/1089493072 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f8958003a10 con 0x7f89681040f0 2024-02-05T07:45:18.705 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.908+0000 7f895f7fe700 1 -- 172.21.15.131:0/1089493072 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8958019610 con 0x7f89681040f0 2024-02-05T07:45:18.705 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.908+0000 7f8970398700 1 -- 172.21.15.131:0/1089493072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896819a6e0 con 0x7f89681040f0 2024-02-05T07:45:18.705 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.909+0000 7f895f7fe700 1 -- 172.21.15.131:0/1089493072 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7f8958019860 con 0x7f89681040f0 2024-02-05T07:45:18.705 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.909+0000 7f8970398700 1 -- 172.21.15.131:0/1089493072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8968063350 con 0x7f89681040f0 2024-02-05T07:45:18.705 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.909+0000 7f895f7fe700 1 --2- 172.21.15.131:0/1089493072 >> v2:172.21.15.131:6800/760045032 conn(0x7f8954038f90 0x7f895403b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:18.705 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.909+0000 7f895f7fe700 1 -- 172.21.15.131:0/1089493072 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f895804d4d0 con 0x7f89681040f0 2024-02-05T07:45:18.706 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.909+0000 7f896d933700 1 --2- 172.21.15.131:0/1089493072 >> v2:172.21.15.131:6800/760045032 conn(0x7f8954038f90 0x7f895403b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:18.706 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.910+0000 7f896d933700 1 --2- 172.21.15.131:0/1089493072 >> v2:172.21.15.131:6800/760045032 conn(0x7f8954038f90 0x7f895403b450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8964006fd0 tx=0x7f8964006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:18.706 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:17.914+0000 7f895f7fe700 1 -- 172.21.15.131:0/1089493072 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8958018710 con 0x7f89681040f0 2024-02-05T07:45:18.706 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:18.117+0000 7f8970398700 1 -- 172.21.15.131:0/1089493072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f896819d180 con 0x7f89681040f0 2024-02-05T07:45:18.706 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:18.117+0000 7f895f7fe700 1 -- 172.21.15.131:0/1089493072 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f8958018350 con 0x7f89681040f0 2024-02-05T07:45:18.706 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:18.119+0000 7f8970398700 1 -- 172.21.15.131:0/1089493072 >> v2:172.21.15.131:6800/760045032 conn(0x7f8954038f90 msgr2=0x7f895403b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:18.706 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:18.119+0000 7f8970398700 1 --2- 172.21.15.131:0/1089493072 >> v2:172.21.15.131:6800/760045032 conn(0x7f8954038f90 0x7f895403b450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8964006fd0 tx=0x7f8964006e40 comp rx=0 tx=0).stop 2024-02-05T07:45:18.706 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:18.119+0000 7f8970398700 1 -- 172.21.15.131:0/1089493072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89681040f0 msgr2=0x7f8968199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:18.706 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:18.119+0000 7f8970398700 1 --2- 172.21.15.131:0/1089493072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89681040f0 0x7f8968199b00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8958006af0 tx=0x7f89580119c0 comp rx=0 tx=0).stop 2024-02-05T07:45:18.706 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:18.119+0000 7f8970398700 1 -- 172.21.15.131:0/1089493072 shutdown_connections 2024-02-05T07:45:18.706 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:18.120+0000 7f8970398700 1 --2- 172.21.15.131:0/1089493072 >> v2:172.21.15.131:6800/760045032 conn(0x7f8954038f90 0x7f895403b450 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:18.706 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:18.120+0000 7f8970398700 1 --2- 172.21.15.131:0/1089493072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89681040f0 0x7f8968199b00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:18.706 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:18.120+0000 7f8970398700 1 -- 172.21.15.131:0/1089493072 >> 172.21.15.131:0/1089493072 conn(0x7f89680ffa40 msgr2=0x7f89681005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:18.706 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:18.120+0000 7f8970398700 1 -- 172.21.15.131:0/1089493072 shutdown_connections 2024-02-05T07:45:18.707 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:18.120+0000 7f8970398700 1 -- 172.21.15.131:0/1089493072 wait complete. 2024-02-05T07:45:20.051 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.168+0000 7f8724248700 1 Processor -- start 2024-02-05T07:45:20.051 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.168+0000 7f8724248700 1 -- start start 2024-02-05T07:45:20.052 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.168+0000 7f8724248700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f871c106930 0x7f871c108d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:20.052 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.168+0000 7f8724248700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c0796e0 con 0x7f871c106930 2024-02-05T07:45:20.052 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.169+0000 7f8721fe4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f871c106930 0x7f871c108d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:20.052 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.169+0000 7f8721fe4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f871c106930 0x7f871c108d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39462/0 (socket says 172.21.15.131:39462) 2024-02-05T07:45:20.052 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.169+0000 7f8721fe4700 1 -- 172.21.15.131:0/3914720435 learned_addr learned my addr 172.21.15.131:0/3914720435 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:20.052 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.169+0000 7f8721fe4700 1 -- 172.21.15.131:0/3914720435 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c109260 con 0x7f871c106930 2024-02-05T07:45:20.052 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.169+0000 7f8721fe4700 1 --2- 172.21.15.131:0/3914720435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f871c106930 0x7f871c108d20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8718009e80 tx=0x7f871800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ab4fc7955988be server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:20.052 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.169+0000 7f8720fe2700 1 -- 172.21.15.131:0/3914720435 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f871800e070 con 0x7f871c106930 2024-02-05T07:45:20.052 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.169+0000 7f8720fe2700 1 -- 172.21.15.131:0/3914720435 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f871800b8c0 con 0x7f871c106930 2024-02-05T07:45:20.052 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.169+0000 7f8720fe2700 1 -- 172.21.15.131:0/3914720435 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87180043d0 con 0x7f871c106930 2024-02-05T07:45:20.052 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.170+0000 7f8724248700 1 -- 172.21.15.131:0/3914720435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f871c106930 msgr2=0x7f871c108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:20.053 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.170+0000 7f8724248700 1 --2- 172.21.15.131:0/3914720435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f871c106930 0x7f871c108d20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8718009e80 tx=0x7f871800b330 comp rx=0 tx=0).stop 2024-02-05T07:45:20.053 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.170+0000 7f8724248700 1 -- 172.21.15.131:0/3914720435 shutdown_connections 2024-02-05T07:45:20.053 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.170+0000 7f8724248700 1 --2- 172.21.15.131:0/3914720435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f871c106930 0x7f871c108d20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:20.053 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.170+0000 7f8724248700 1 -- 172.21.15.131:0/3914720435 >> 172.21.15.131:0/3914720435 conn(0x7f871c100300 msgr2=0x7f871c102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:20.053 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.170+0000 7f8724248700 1 -- 172.21.15.131:0/3914720435 shutdown_connections 2024-02-05T07:45:20.053 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.170+0000 7f8724248700 1 -- 172.21.15.131:0/3914720435 wait complete. 2024-02-05T07:45:20.053 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.171+0000 7f8724248700 1 Processor -- start 2024-02-05T07:45:20.053 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.171+0000 7f8724248700 1 -- start start 2024-02-05T07:45:20.053 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.171+0000 7f8724248700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f871c106930 0x7f871c19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:20.053 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.171+0000 7f8724248700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c19c270 con 0x7f871c106930 2024-02-05T07:45:20.053 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.171+0000 7f8721fe4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f871c106930 0x7f871c19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:20.053 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.171+0000 7f8721fe4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f871c106930 0x7f871c19bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39470/0 (socket says 172.21.15.131:39470) 2024-02-05T07:45:20.053 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.171+0000 7f8721fe4700 1 -- 172.21.15.131:0/3273606395 learned_addr learned my addr 172.21.15.131:0/3273606395 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:20.053 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.171+0000 7f8721fe4700 1 -- 172.21.15.131:0/3273606395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8718009b30 con 0x7f871c106930 2024-02-05T07:45:20.053 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.171+0000 7f8721fe4700 1 --2- 172.21.15.131:0/3273606395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f871c106930 0x7f871c19bd30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f87180041e0 tx=0x7f8718003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:20.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.172+0000 7f8712ffd700 1 -- 172.21.15.131:0/3273606395 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f871800e040 con 0x7f871c106930 2024-02-05T07:45:20.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.172+0000 7f8712ffd700 1 -- 172.21.15.131:0/3273606395 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f871800ba30 con 0x7f871c106930 2024-02-05T07:45:20.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.172+0000 7f8724248700 1 -- 172.21.15.131:0/3273606395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871c19c470 con 0x7f871c106930 2024-02-05T07:45:20.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.172+0000 7f8712ffd700 1 -- 172.21.15.131:0/3273606395 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87180125c0 con 0x7f871c106930 2024-02-05T07:45:20.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.172+0000 7f8724248700 1 -- 172.21.15.131:0/3273606395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c19c970 con 0x7f871c106930 2024-02-05T07:45:20.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.172+0000 7f8724248700 1 -- 172.21.15.131:0/3273606395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871c195cc0 con 0x7f871c106930 2024-02-05T07:45:20.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.173+0000 7f8712ffd700 1 -- 172.21.15.131:0/3273606395 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7f871801c650 con 0x7f871c106930 2024-02-05T07:45:20.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.173+0000 7f8712ffd700 1 --2- 172.21.15.131:0/3273606395 >> v2:172.21.15.131:6800/760045032 conn(0x7f8708038f40 0x7f870803b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:20.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.173+0000 7f8712ffd700 1 -- 172.21.15.131:0/3273606395 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f871804d160 con 0x7f871c106930 2024-02-05T07:45:20.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.173+0000 7f87217e3700 1 --2- 172.21.15.131:0/3273606395 >> v2:172.21.15.131:6800/760045032 conn(0x7f8708038f40 0x7f870803b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:20.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.173+0000 7f87217e3700 1 --2- 172.21.15.131:0/3273606395 >> v2:172.21.15.131:6800/760045032 conn(0x7f8708038f40 0x7f870803b400 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f870c006fd0 tx=0x7f870c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:20.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.178+0000 7f8712ffd700 1 -- 172.21.15.131:0/3273606395 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f871801ba90 con 0x7f871c106930 2024-02-05T07:45:20.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.475+0000 7f8724248700 1 -- 172.21.15.131:0/3273606395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f871c105370 con 0x7f871c106930 2024-02-05T07:45:20.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.478+0000 7f8712ffd700 1 -- 172.21.15.131:0/3273606395 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f8718020020 con 0x7f871c106930 2024-02-05T07:45:20.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.480+0000 7f8724248700 1 -- 172.21.15.131:0/3273606395 >> v2:172.21.15.131:6800/760045032 conn(0x7f8708038f40 msgr2=0x7f870803b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:20.054 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.480+0000 7f8724248700 1 --2- 172.21.15.131:0/3273606395 >> v2:172.21.15.131:6800/760045032 conn(0x7f8708038f40 0x7f870803b400 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f870c006fd0 tx=0x7f870c006e40 comp rx=0 tx=0).stop 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.480+0000 7f8724248700 1 -- 172.21.15.131:0/3273606395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f871c106930 msgr2=0x7f871c19bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.480+0000 7f8724248700 1 --2- 172.21.15.131:0/3273606395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f871c106930 0x7f871c19bd30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f87180041e0 tx=0x7f8718003d00 comp rx=0 tx=0).stop 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.480+0000 7f8724248700 1 -- 172.21.15.131:0/3273606395 shutdown_connections 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.480+0000 7f8724248700 1 --2- 172.21.15.131:0/3273606395 >> v2:172.21.15.131:6800/760045032 conn(0x7f8708038f40 0x7f870803b400 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.480+0000 7f8724248700 1 --2- 172.21.15.131:0/3273606395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f871c106930 0x7f871c19bd30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.480+0000 7f8724248700 1 -- 172.21.15.131:0/3273606395 >> 172.21.15.131:0/3273606395 conn(0x7f871c100300 msgr2=0x7f871c100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.480+0000 7f8724248700 1 -- 172.21.15.131:0/3273606395 shutdown_connections 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-02-05T07:45:19.480+0000 7f8724248700 1 -- 172.21.15.131:0/3273606395 wait complete. 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout: sudo /sbin/cephadm shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout:Or, if you are only running a single cluster on this host: 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout: sudo /sbin/cephadm shell 2024-02-05T07:45:20.055 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:45:20.056 INFO:teuthology.orchestra.run.smithi131.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-05T07:45:20.056 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:45:20.056 INFO:teuthology.orchestra.run.smithi131.stdout: ceph telemetry on 2024-02-05T07:45:20.056 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:45:20.056 INFO:teuthology.orchestra.run.smithi131.stdout:For more information see: 2024-02-05T07:45:20.056 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:45:20.056 INFO:teuthology.orchestra.run.smithi131.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-05T07:45:20.056 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:45:20.056 INFO:teuthology.orchestra.run.smithi131.stdout:Bootstrap complete. 2024-02-05T07:45:20.093 INFO:tasks.cephadm:Fetching config... 2024-02-05T07:45:20.093 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:45:20.093 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-05T07:45:20.109 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-05T07:45:20.109 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:45:20.109 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-05T07:45:20.165 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-05T07:45:20.165 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:45:20.165 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-05T07:45:20.231 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-05T07:45:20.232 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:45:20.232 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-05T07:45:20.287 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-05T07:45:20.287 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6kv1YeFWAV37cFlbLKm3kh5Y1F8XrvmDb89VWrFCVhuUA43fHh+iQTpx5Xt7kgD6K1jx5p6ja8gGnh29ul4O3hsliYWNL+wehjYy55GafZRdQmYHcCCcyg6ViN/rkC05v8K1HpGisqU1A6NhjRrC6yEPWVmASx+ejM7KBaADk+aa2Sl7nVVX2olZXveaNzMD1rnVIdE1eXySPvcvwQc1unYotafdm70wXJAWgoLFWQ2FKsDiXv2A9o/GR0JSzSBqSBGjuhYj0qPBLNb+86PFVnD05jhTmTrpK+BghKS0N19jCBjmiiLPC3LIjqgURBBHb/F29Pl8rOFm0KjoGUlx2Y7mGQjRweI6XSwZghGmKJZ6WBRGZwMffP4x6GZNvNaEnE8Tjck1xqKq3xvNLJ/28kuEPEA91+3Q5Cj6rYnsos8Bt2i8RO5kNW6oHLHMcAS/WhLOdSfSAmxhA+f84OWrxno+M4xBK0PX92ZQpbS6nRreoZaS2DmRKPDIWzS1RBLE= ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-05T07:45:20.379 INFO:teuthology.orchestra.run.smithi131.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6kv1YeFWAV37cFlbLKm3kh5Y1F8XrvmDb89VWrFCVhuUA43fHh+iQTpx5Xt7kgD6K1jx5p6ja8gGnh29ul4O3hsliYWNL+wehjYy55GafZRdQmYHcCCcyg6ViN/rkC05v8K1HpGisqU1A6NhjRrC6yEPWVmASx+ejM7KBaADk+aa2Sl7nVVX2olZXveaNzMD1rnVIdE1eXySPvcvwQc1unYotafdm70wXJAWgoLFWQ2FKsDiXv2A9o/GR0JSzSBqSBGjuhYj0qPBLNb+86PFVnD05jhTmTrpK+BghKS0N19jCBjmiiLPC3LIjqgURBBHb/F29Pl8rOFm0KjoGUlx2Y7mGQjRweI6XSwZghGmKJZ6WBRGZwMffP4x6GZNvNaEnE8Tjck1xqKq3xvNLJ/28kuEPEA91+3Q5Cj6rYnsos8Bt2i8RO5kNW6oHLHMcAS/WhLOdSfSAmxhA+f84OWrxno+M4xBK0PX92ZQpbS6nRreoZaS2DmRKPDIWzS1RBLE= ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:45:20.393 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6kv1YeFWAV37cFlbLKm3kh5Y1F8XrvmDb89VWrFCVhuUA43fHh+iQTpx5Xt7kgD6K1jx5p6ja8gGnh29ul4O3hsliYWNL+wehjYy55GafZRdQmYHcCCcyg6ViN/rkC05v8K1HpGisqU1A6NhjRrC6yEPWVmASx+ejM7KBaADk+aa2Sl7nVVX2olZXveaNzMD1rnVIdE1eXySPvcvwQc1unYotafdm70wXJAWgoLFWQ2FKsDiXv2A9o/GR0JSzSBqSBGjuhYj0qPBLNb+86PFVnD05jhTmTrpK+BghKS0N19jCBjmiiLPC3LIjqgURBBHb/F29Pl8rOFm0KjoGUlx2Y7mGQjRweI6XSwZghGmKJZ6WBRGZwMffP4x6GZNvNaEnE8Tjck1xqKq3xvNLJ/28kuEPEA91+3Q5Cj6rYnsos8Bt2i8RO5kNW6oHLHMcAS/WhLOdSfSAmxhA+f84OWrxno+M4xBK0PX92ZQpbS6nRreoZaS2DmRKPDIWzS1RBLE= ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-05T07:45:20.439 INFO:teuthology.orchestra.run.smithi157.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6kv1YeFWAV37cFlbLKm3kh5Y1F8XrvmDb89VWrFCVhuUA43fHh+iQTpx5Xt7kgD6K1jx5p6ja8gGnh29ul4O3hsliYWNL+wehjYy55GafZRdQmYHcCCcyg6ViN/rkC05v8K1HpGisqU1A6NhjRrC6yEPWVmASx+ejM7KBaADk+aa2Sl7nVVX2olZXveaNzMD1rnVIdE1eXySPvcvwQc1unYotafdm70wXJAWgoLFWQ2FKsDiXv2A9o/GR0JSzSBqSBGjuhYj0qPBLNb+86PFVnD05jhTmTrpK+BghKS0N19jCBjmiiLPC3LIjqgURBBHb/F29Pl8rOFm0KjoGUlx2Y7mGQjRweI6XSwZghGmKJZ6WBRGZwMffP4x6GZNvNaEnE8Tjck1xqKq3xvNLJ/28kuEPEA91+3Q5Cj6rYnsos8Bt2i8RO5kNW6oHLHMcAS/WhLOdSfSAmxhA+f84OWrxno+M4xBK0PX92ZQpbS6nRreoZaS2DmRKPDIWzS1RBLE= ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:45:20.454 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-05T07:45:20.691 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:45:20.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:20 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3273606395' entity='client.admin' 2024-02-05T07:45:23.359 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:23 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:23.359 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:23 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:23.359 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:23 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.359 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:23 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:23.359 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:23 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:45:23.360 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:23 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:23.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.466+0000 7fc15c66c700 1 -- 172.21.15.131:0/1996806965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1541006c0 msgr2=0x7fc154100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:23.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.466+0000 7fc15c66c700 1 --2- 172.21.15.131:0/1996806965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1541006c0 0x7fc154100ae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc150009910 tx=0x7fc150009c20 comp rx=0 tx=0).stop 2024-02-05T07:45:23.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.466+0000 7fc15c66c700 1 -- 172.21.15.131:0/1996806965 shutdown_connections 2024-02-05T07:45:23.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.466+0000 7fc15c66c700 1 --2- 172.21.15.131:0/1996806965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1541006c0 0x7fc154100ae0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:23.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.466+0000 7fc15c66c700 1 -- 172.21.15.131:0/1996806965 >> 172.21.15.131:0/1996806965 conn(0x7fc1540fbc80 msgr2=0x7fc1540fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:23.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.466+0000 7fc15c66c700 1 -- 172.21.15.131:0/1996806965 shutdown_connections 2024-02-05T07:45:23.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.466+0000 7fc15c66c700 1 -- 172.21.15.131:0/1996806965 wait complete. 2024-02-05T07:45:23.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.467+0000 7fc15c66c700 1 Processor -- start 2024-02-05T07:45:23.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.467+0000 7fc15c66c700 1 -- start start 2024-02-05T07:45:23.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.467+0000 7fc15c66c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1541006c0 0x7fc1540fdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:23.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.467+0000 7fc15c66c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1540fe030 con 0x7fc1541006c0 2024-02-05T07:45:23.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.467+0000 7fc15a408700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1541006c0 0x7fc1540fdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:23.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.467+0000 7fc15a408700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1541006c0 0x7fc1540fdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39490/0 (socket says 172.21.15.131:39490) 2024-02-05T07:45:23.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.467+0000 7fc15a408700 1 -- 172.21.15.131:0/4143970759 learned_addr learned my addr 172.21.15.131:0/4143970759 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:23.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.468+0000 7fc15a408700 1 -- 172.21.15.131:0/4143970759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1500095f0 con 0x7fc1541006c0 2024-02-05T07:45:23.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.468+0000 7fc15a408700 1 --2- 172.21.15.131:0/4143970759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1541006c0 0x7fc1540fdaf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc1500057a0 tx=0x7fc150004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:23.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.468+0000 7fc14b7fe700 1 -- 172.21.15.131:0/4143970759 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc15001d070 con 0x7fc1541006c0 2024-02-05T07:45:23.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.468+0000 7fc14b7fe700 1 -- 172.21.15.131:0/4143970759 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc150022470 con 0x7fc1541006c0 2024-02-05T07:45:23.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.468+0000 7fc14b7fe700 1 -- 172.21.15.131:0/4143970759 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc15000f5f0 con 0x7fc1541006c0 2024-02-05T07:45:23.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.468+0000 7fc15c66c700 1 -- 172.21.15.131:0/4143970759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1540fc220 con 0x7fc1541006c0 2024-02-05T07:45:23.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.468+0000 7fc15c66c700 1 -- 172.21.15.131:0/4143970759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1540fc6c0 con 0x7fc1541006c0 2024-02-05T07:45:23.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.469+0000 7fc14b7fe700 1 -- 172.21.15.131:0/4143970759 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7fc15000ba90 con 0x7fc1541006c0 2024-02-05T07:45:23.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.469+0000 7fc14b7fe700 1 --2- 172.21.15.131:0/4143970759 >> v2:172.21.15.131:6800/760045032 conn(0x7fc140038f90 0x7fc14003b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:23.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.469+0000 7fc15c66c700 1 -- 172.21.15.131:0/4143970759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc154063350 con 0x7fc1541006c0 2024-02-05T07:45:23.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.469+0000 7fc14b7fe700 1 -- 172.21.15.131:0/4143970759 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fc15004d1d0 con 0x7fc1541006c0 2024-02-05T07:45:23.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.469+0000 7fc159c07700 1 --2- 172.21.15.131:0/4143970759 >> v2:172.21.15.131:6800/760045032 conn(0x7fc140038f90 0x7fc14003b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:23.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.470+0000 7fc159c07700 1 --2- 172.21.15.131:0/4143970759 >> v2:172.21.15.131:6800/760045032 conn(0x7fc140038f90 0x7fc14003b450 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc144006fd0 tx=0x7fc144006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:23.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.477+0000 7fc14b7fe700 1 -- 172.21.15.131:0/4143970759 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1500228c0 con 0x7fc1541006c0 2024-02-05T07:45:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.678+0000 7fc15c66c700 1 -- 172.21.15.131:0/4143970759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fc154105b90 con 0x7fc1541006c0 2024-02-05T07:45:23.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.681+0000 7fc14b7fe700 1 -- 172.21.15.131:0/4143970759 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fc15002bc40 con 0x7fc1541006c0 2024-02-05T07:45:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.683+0000 7fc15c66c700 1 -- 172.21.15.131:0/4143970759 >> v2:172.21.15.131:6800/760045032 conn(0x7fc140038f90 msgr2=0x7fc14003b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.683+0000 7fc15c66c700 1 --2- 172.21.15.131:0/4143970759 >> v2:172.21.15.131:6800/760045032 conn(0x7fc140038f90 0x7fc14003b450 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc144006fd0 tx=0x7fc144006e40 comp rx=0 tx=0).stop 2024-02-05T07:45:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.683+0000 7fc15c66c700 1 -- 172.21.15.131:0/4143970759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1541006c0 msgr2=0x7fc1540fdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.683+0000 7fc15c66c700 1 --2- 172.21.15.131:0/4143970759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1541006c0 0x7fc1540fdaf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc1500057a0 tx=0x7fc150004f20 comp rx=0 tx=0).stop 2024-02-05T07:45:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.683+0000 7fc15c66c700 1 -- 172.21.15.131:0/4143970759 shutdown_connections 2024-02-05T07:45:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.683+0000 7fc15c66c700 1 --2- 172.21.15.131:0/4143970759 >> v2:172.21.15.131:6800/760045032 conn(0x7fc140038f90 0x7fc14003b450 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.683+0000 7fc15c66c700 1 --2- 172.21.15.131:0/4143970759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1541006c0 0x7fc1540fdaf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.683+0000 7fc15c66c700 1 -- 172.21.15.131:0/4143970759 >> 172.21.15.131:0/4143970759 conn(0x7fc1540fbc80 msgr2=0x7fc1541042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.683+0000 7fc15c66c700 1 -- 172.21.15.131:0/4143970759 shutdown_connections 2024-02-05T07:45:23.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:23.683+0000 7fc15c66c700 1 -- 172.21.15.131:0/4143970759 wait complete. 2024-02-05T07:45:24.348 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-05T07:45:24.348 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-05T07:45:24.765 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:45:24.998 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4143970759' entity='client.admin' 2024-02-05T07:45:24.998 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:24 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:45:24.998 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:24 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:26.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.283+0000 7fc4c5330700 1 -- 172.21.15.131:0/3451968522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4c0100660 msgr2=0x7fc4c0100a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:26.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.283+0000 7fc4c5330700 1 --2- 172.21.15.131:0/3451968522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4c0100660 0x7fc4c0100a80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009910 tx=0x7fc4b0009c20 comp rx=0 tx=0).stop 2024-02-05T07:45:26.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.283+0000 7fc4c5330700 1 -- 172.21.15.131:0/3451968522 shutdown_connections 2024-02-05T07:45:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.283+0000 7fc4c5330700 1 --2- 172.21.15.131:0/3451968522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4c0100660 0x7fc4c0100a80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.283+0000 7fc4c5330700 1 -- 172.21.15.131:0/3451968522 >> 172.21.15.131:0/3451968522 conn(0x7fc4c00fbc20 msgr2=0x7fc4c00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.283+0000 7fc4c5330700 1 -- 172.21.15.131:0/3451968522 shutdown_connections 2024-02-05T07:45:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.283+0000 7fc4c5330700 1 -- 172.21.15.131:0/3451968522 wait complete. 2024-02-05T07:45:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.283+0000 7fc4c5330700 1 Processor -- start 2024-02-05T07:45:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.284+0000 7fc4c5330700 1 -- start start 2024-02-05T07:45:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.284+0000 7fc4c5330700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4c0100660 0x7fc4c0196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.284+0000 7fc4c5330700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c0196940 con 0x7fc4c0100660 2024-02-05T07:45:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.284+0000 7fc4beffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4c0100660 0x7fc4c0196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.284+0000 7fc4beffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4c0100660 0x7fc4c0196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39498/0 (socket says 172.21.15.131:39498) 2024-02-05T07:45:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.284+0000 7fc4beffd700 1 -- 172.21.15.131:0/2540983162 learned_addr learned my addr 172.21.15.131:0/2540983162 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.284+0000 7fc4beffd700 1 -- 172.21.15.131:0/2540983162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b00095f0 con 0x7fc4c0100660 2024-02-05T07:45:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.284+0000 7fc4beffd700 1 --2- 172.21.15.131:0/2540983162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4c0100660 0x7fc4c0196400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0004f40 tx=0x7fc4b0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:26.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.284+0000 7fc4bcff9700 1 -- 172.21.15.131:0/2540983162 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4b001d070 con 0x7fc4c0100660 2024-02-05T07:45:26.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.284+0000 7fc4bcff9700 1 -- 172.21.15.131:0/2540983162 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc4b000be10 con 0x7fc4c0100660 2024-02-05T07:45:26.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.284+0000 7fc4c5330700 1 -- 172.21.15.131:0/2540983162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c0196b40 con 0x7fc4c0100660 2024-02-05T07:45:26.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.285+0000 7fc4bcff9700 1 -- 172.21.15.131:0/2540983162 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4b0022da0 con 0x7fc4c0100660 2024-02-05T07:45:26.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.285+0000 7fc4c5330700 1 -- 172.21.15.131:0/2540983162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c0196fe0 con 0x7fc4c0100660 2024-02-05T07:45:26.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.285+0000 7fc4bcff9700 1 -- 172.21.15.131:0/2540983162 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7fc4b000f460 con 0x7fc4c0100660 2024-02-05T07:45:26.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.285+0000 7fc4c5330700 1 -- 172.21.15.131:0/2540983162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c0063350 con 0x7fc4c0100660 2024-02-05T07:45:26.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.286+0000 7fc4bcff9700 1 --2- 172.21.15.131:0/2540983162 >> v2:172.21.15.131:6800/760045032 conn(0x7fc4ac038f40 0x7fc4ac03b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:26.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.286+0000 7fc4bcff9700 1 -- 172.21.15.131:0/2540983162 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fc4b004d1a0 con 0x7fc4c0100660 2024-02-05T07:45:26.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.286+0000 7fc4b67fc700 1 --2- 172.21.15.131:0/2540983162 >> v2:172.21.15.131:6800/760045032 conn(0x7fc4ac038f40 0x7fc4ac03b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:26.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.286+0000 7fc4b67fc700 1 --2- 172.21.15.131:0/2540983162 >> v2:172.21.15.131:6800/760045032 conn(0x7fc4ac038f40 0x7fc4ac03b400 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8006fd0 tx=0x7fc4a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:26.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.290+0000 7fc4bcff9700 1 -- 172.21.15.131:0/2540983162 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc4b0020960 con 0x7fc4c0100660 2024-02-05T07:45:26.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.488+0000 7fc4c5330700 1 -- 172.21.15.131:0/2540983162 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4c006a2a0 con 0x7fc4ac038f40 2024-02-05T07:45:26.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.491+0000 7fc4bcff9700 1 -- 172.21.15.131:0/2540983162 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc4c006a2a0 con 0x7fc4ac038f40 2024-02-05T07:45:26.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.493+0000 7fc4c5330700 1 -- 172.21.15.131:0/2540983162 >> v2:172.21.15.131:6800/760045032 conn(0x7fc4ac038f40 msgr2=0x7fc4ac03b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:26.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.493+0000 7fc4c5330700 1 --2- 172.21.15.131:0/2540983162 >> v2:172.21.15.131:6800/760045032 conn(0x7fc4ac038f40 0x7fc4ac03b400 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8006fd0 tx=0x7fc4a8006e40 comp rx=0 tx=0).stop 2024-02-05T07:45:26.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.493+0000 7fc4c5330700 1 -- 172.21.15.131:0/2540983162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4c0100660 msgr2=0x7fc4c0196400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:26.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.493+0000 7fc4c5330700 1 --2- 172.21.15.131:0/2540983162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4c0100660 0x7fc4c0196400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0004f40 tx=0x7fc4b0005020 comp rx=0 tx=0).stop 2024-02-05T07:45:26.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.493+0000 7fc4c5330700 1 -- 172.21.15.131:0/2540983162 shutdown_connections 2024-02-05T07:45:26.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.493+0000 7fc4c5330700 1 --2- 172.21.15.131:0/2540983162 >> v2:172.21.15.131:6800/760045032 conn(0x7fc4ac038f40 0x7fc4ac03b400 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:26.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.493+0000 7fc4c5330700 1 --2- 172.21.15.131:0/2540983162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4c0100660 0x7fc4c0196400 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:26.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.493+0000 7fc4c5330700 1 -- 172.21.15.131:0/2540983162 >> 172.21.15.131:0/2540983162 conn(0x7fc4c00fbc20 msgr2=0x7fc4c0069b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:26.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.493+0000 7fc4c5330700 1 -- 172.21.15.131:0/2540983162 shutdown_connections 2024-02-05T07:45:26.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:26.493+0000 7fc4c5330700 1 -- 172.21.15.131:0/2540983162 wait complete. 2024-02-05T07:45:27.362 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi157 2024-02-05T07:45:27.362 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-02-05T07:45:27.362 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.conf 2024-02-05T07:45:27.382 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-02-05T07:45:27.382 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:45:27.441 INFO:tasks.cephadm:Adding host smithi157 to orchestrator... 2024-02-05T07:45:27.441 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch host add smithi157 2024-02-05T07:45:27.673 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:45:27.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:27 smithi131 ceph-mon[29507]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:45:27.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:27 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:27.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:27 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:45:27.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:27 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:45:27.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:27 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:45:27.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:27 smithi131 ceph-mon[29507]: Updating smithi131:/etc/ceph/ceph.conf 2024-02-05T07:45:27.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:27 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:27.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:27 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:27.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:27 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:28.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:28 smithi131 ceph-mon[29507]: Updating smithi131:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:45:28.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:28 smithi131 ceph-mon[29507]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:45:28.993 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:28 smithi131 ceph-mon[29507]: Updating smithi131:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:45:29.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.411+0000 7ff7bad9b700 1 -- 172.21.15.131:0/845070395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b41055c0 msgr2=0x7ff7b41059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:29.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.411+0000 7ff7bad9b700 1 --2- 172.21.15.131:0/845070395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b41055c0 0x7ff7b41059a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400b0a0 tx=0x7ff7a400b3b0 comp rx=0 tx=0).stop 2024-02-05T07:45:29.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.411+0000 7ff7bad9b700 1 -- 172.21.15.131:0/845070395 shutdown_connections 2024-02-05T07:45:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.411+0000 7ff7bad9b700 1 --2- 172.21.15.131:0/845070395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b41055c0 0x7ff7b41059a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.411+0000 7ff7bad9b700 1 -- 172.21.15.131:0/845070395 >> 172.21.15.131:0/845070395 conn(0x7ff7b40fb860 msgr2=0x7ff7b40fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.411+0000 7ff7bad9b700 1 -- 172.21.15.131:0/845070395 shutdown_connections 2024-02-05T07:45:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.411+0000 7ff7bad9b700 1 -- 172.21.15.131:0/845070395 wait complete. 2024-02-05T07:45:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.411+0000 7ff7bad9b700 1 Processor -- start 2024-02-05T07:45:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.411+0000 7ff7bad9b700 1 -- start start 2024-02-05T07:45:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.412+0000 7ff7bad9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b41055c0 0x7ff7b41951e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.412+0000 7ff7bad9b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b4100a80 con 0x7ff7b41055c0 2024-02-05T07:45:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.412+0000 7ff7b8b37700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b41055c0 0x7ff7b41951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.412+0000 7ff7b8b37700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b41055c0 0x7ff7b41951e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:32828/0 (socket says 172.21.15.131:32828) 2024-02-05T07:45:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.412+0000 7ff7b8b37700 1 -- 172.21.15.131:0/1149775154 learned_addr learned my addr 172.21.15.131:0/1149775154 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.412+0000 7ff7b8b37700 1 -- 172.21.15.131:0/1149775154 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a4009d00 con 0x7ff7b41055c0 2024-02-05T07:45:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.412+0000 7ff7b8b37700 1 --2- 172.21.15.131:0/1149775154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b41055c0 0x7ff7b41951e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4015040 tx=0x7ff7a4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.412+0000 7ff7b1ffb700 1 -- 172.21.15.131:0/1149775154 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7a400e040 con 0x7ff7b41055c0 2024-02-05T07:45:29.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.412+0000 7ff7b1ffb700 1 -- 172.21.15.131:0/1149775154 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7ff7a401db10 con 0x7ff7b41055c0 2024-02-05T07:45:29.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.412+0000 7ff7bad9b700 1 -- 172.21.15.131:0/1149775154 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b4195720 con 0x7ff7b41055c0 2024-02-05T07:45:29.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.412+0000 7ff7b1ffb700 1 -- 172.21.15.131:0/1149775154 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7a40044f0 con 0x7ff7b41055c0 2024-02-05T07:45:29.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.412+0000 7ff7bad9b700 1 -- 172.21.15.131:0/1149775154 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b4195ba0 con 0x7ff7b41055c0 2024-02-05T07:45:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.413+0000 7ff7b1ffb700 1 -- 172.21.15.131:0/1149775154 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7ff7a4019070 con 0x7ff7b41055c0 2024-02-05T07:45:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.413+0000 7ff7bad9b700 1 -- 172.21.15.131:0/1149775154 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b4104760 con 0x7ff7b41055c0 2024-02-05T07:45:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.413+0000 7ff7b1ffb700 1 --2- 172.21.15.131:0/1149775154 >> v2:172.21.15.131:6800/760045032 conn(0x7ff7a0038f90 0x7ff7a003b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.413+0000 7ff7b1ffb700 1 -- 172.21.15.131:0/1149775154 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7ff7a404d910 con 0x7ff7b41055c0 2024-02-05T07:45:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.414+0000 7ff7b3fff700 1 --2- 172.21.15.131:0/1149775154 >> v2:172.21.15.131:6800/760045032 conn(0x7ff7a0038f90 0x7ff7a003b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:29.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.414+0000 7ff7b3fff700 1 --2- 172.21.15.131:0/1149775154 >> v2:172.21.15.131:6800/760045032 conn(0x7ff7a0038f90 0x7ff7a003b450 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff79c006fd0 tx=0x7ff79c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:29.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.418+0000 7ff7b1ffb700 1 -- 172.21.15.131:0/1149775154 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff7a401a380 con 0x7ff7b41055c0 2024-02-05T07:45:29.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:29.616+0000 7ff7bad9b700 1 -- 172.21.15.131:0/1149775154 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7b4106020 con 0x7ff7a0038f90 2024-02-05T07:45:30.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:29 smithi131 ceph-mon[29507]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:45:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:30 smithi131 ceph-mon[29507]: Deploying cephadm binary to smithi157 2024-02-05T07:45:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:30 smithi131 ceph-mon[29507]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:33.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:33.918+0000 7ff7b1ffb700 1 -- 172.21.15.131:0/1149775154 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ff7b4106020 con 0x7ff7a0038f90 2024-02-05T07:45:33.920 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:33 smithi131 ceph-mon[29507]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:33.921 INFO:teuthology.orchestra.run.smithi131.stdout:Added host 'smithi157' with addr '172.21.15.157' 2024-02-05T07:45:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:33.920+0000 7ff7bad9b700 1 -- 172.21.15.131:0/1149775154 >> v2:172.21.15.131:6800/760045032 conn(0x7ff7a0038f90 msgr2=0x7ff7a003b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:33.920+0000 7ff7bad9b700 1 --2- 172.21.15.131:0/1149775154 >> v2:172.21.15.131:6800/760045032 conn(0x7ff7a0038f90 0x7ff7a003b450 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff79c006fd0 tx=0x7ff79c006e40 comp rx=0 tx=0).stop 2024-02-05T07:45:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:33.920+0000 7ff7bad9b700 1 -- 172.21.15.131:0/1149775154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b41055c0 msgr2=0x7ff7b41951e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:33.920+0000 7ff7bad9b700 1 --2- 172.21.15.131:0/1149775154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b41055c0 0x7ff7b41951e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4015040 tx=0x7ff7a4003ce0 comp rx=0 tx=0).stop 2024-02-05T07:45:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:33.921+0000 7ff7bad9b700 1 -- 172.21.15.131:0/1149775154 shutdown_connections 2024-02-05T07:45:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:33.921+0000 7ff7bad9b700 1 --2- 172.21.15.131:0/1149775154 >> v2:172.21.15.131:6800/760045032 conn(0x7ff7a0038f90 0x7ff7a003b450 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:33.921+0000 7ff7bad9b700 1 --2- 172.21.15.131:0/1149775154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7b41055c0 0x7ff7b41951e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:33.921+0000 7ff7bad9b700 1 -- 172.21.15.131:0/1149775154 >> 172.21.15.131:0/1149775154 conn(0x7ff7b40fb860 msgr2=0x7ff7b40fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:33.921+0000 7ff7bad9b700 1 -- 172.21.15.131:0/1149775154 shutdown_connections 2024-02-05T07:45:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:33.921+0000 7ff7bad9b700 1 -- 172.21.15.131:0/1149775154 wait complete. 2024-02-05T07:45:34.564 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch host ls --format=json 2024-02-05T07:45:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:45:35.182 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:34 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:35.182 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:34 smithi131 ceph-mon[29507]: Added host smithi157 2024-02-05T07:45:35.182 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:34 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:45:35.183 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:34 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:35.183 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:34 smithi131 ceph-mon[29507]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:36.196 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:36.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.405+0000 7f40d49b8700 1 -- 172.21.15.131:0/1009599959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d01076d0 msgr2=0x7f40d0107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:36.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.405+0000 7f40d49b8700 1 --2- 172.21.15.131:0/1009599959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d01076d0 0x7f40d0107ab0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009910 tx=0x7f40c0009c20 comp rx=0 tx=0).stop 2024-02-05T07:45:36.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.405+0000 7f40d49b8700 1 -- 172.21.15.131:0/1009599959 shutdown_connections 2024-02-05T07:45:36.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.405+0000 7f40d49b8700 1 --2- 172.21.15.131:0/1009599959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d01076d0 0x7f40d0107ab0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:36.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.405+0000 7f40d49b8700 1 -- 172.21.15.131:0/1009599959 >> 172.21.15.131:0/1009599959 conn(0x7f40d00fd200 msgr2=0x7f40d00ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:36.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.405+0000 7f40d49b8700 1 -- 172.21.15.131:0/1009599959 shutdown_connections 2024-02-05T07:45:36.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.405+0000 7f40d49b8700 1 -- 172.21.15.131:0/1009599959 wait complete. 2024-02-05T07:45:36.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.406+0000 7f40d49b8700 1 Processor -- start 2024-02-05T07:45:36.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.406+0000 7f40d49b8700 1 -- start start 2024-02-05T07:45:36.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.406+0000 7f40d49b8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d01076d0 0x7f40d0196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:36.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.406+0000 7f40d49b8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d0196c50 con 0x7f40d01076d0 2024-02-05T07:45:36.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.407+0000 7f40ce59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d01076d0 0x7f40d0196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:36.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.407+0000 7f40ce59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d01076d0 0x7f40d0196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:32840/0 (socket says 172.21.15.131:32840) 2024-02-05T07:45:36.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.407+0000 7f40ce59c700 1 -- 172.21.15.131:0/3412626374 learned_addr learned my addr 172.21.15.131:0/3412626374 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:36.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.407+0000 7f40ce59c700 1 -- 172.21.15.131:0/3412626374 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c00095f0 con 0x7f40d01076d0 2024-02-05T07:45:36.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.407+0000 7f40ce59c700 1 --2- 172.21.15.131:0/3412626374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d01076d0 0x7f40d0196710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f40c000f690 tx=0x7f40c000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:36.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.407+0000 7f40bf7fe700 1 -- 172.21.15.131:0/3412626374 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40c001d070 con 0x7f40d01076d0 2024-02-05T07:45:36.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.407+0000 7f40d49b8700 1 -- 172.21.15.131:0/3412626374 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d0196eb0 con 0x7f40d01076d0 2024-02-05T07:45:36.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.407+0000 7f40bf7fe700 1 -- 172.21.15.131:0/3412626374 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f40c0004500 con 0x7f40d01076d0 2024-02-05T07:45:36.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.407+0000 7f40bf7fe700 1 -- 172.21.15.131:0/3412626374 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40c0003b30 con 0x7f40d01076d0 2024-02-05T07:45:36.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.407+0000 7f40d49b8700 1 -- 172.21.15.131:0/3412626374 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d0197330 con 0x7f40d01076d0 2024-02-05T07:45:36.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.408+0000 7f40bf7fe700 1 -- 172.21.15.131:0/3412626374 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7f40c002b440 con 0x7f40d01076d0 2024-02-05T07:45:36.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.408+0000 7f40d49b8700 1 -- 172.21.15.131:0/3412626374 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d0063350 con 0x7f40d01076d0 2024-02-05T07:45:36.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.408+0000 7f40bf7fe700 1 --2- 172.21.15.131:0/3412626374 >> v2:172.21.15.131:6800/760045032 conn(0x7f40b8038f40 0x7f40b803b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:36.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.408+0000 7f40bf7fe700 1 -- 172.21.15.131:0/3412626374 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f40c004cea0 con 0x7f40d01076d0 2024-02-05T07:45:36.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.408+0000 7f40cdd9b700 1 --2- 172.21.15.131:0/3412626374 >> v2:172.21.15.131:6800/760045032 conn(0x7f40b8038f40 0x7f40b803b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:36.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.409+0000 7f40cdd9b700 1 --2- 172.21.15.131:0/3412626374 >> v2:172.21.15.131:6800/760045032 conn(0x7f40b8038f40 0x7f40b803b400 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f40c4006fd0 tx=0x7f40c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:36.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.413+0000 7f40bf7fe700 1 -- 172.21.15.131:0/3412626374 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f40c0015350 con 0x7f40d01076d0 2024-02-05T07:45:36.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.615+0000 7f40d49b8700 1 -- 172.21.15.131:0/3412626374 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40d010b250 con 0x7f40b8038f40 2024-02-05T07:45:36.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.616+0000 7f40bf7fe700 1 -- 172.21.15.131:0/3412626374 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f40d010b250 con 0x7f40b8038f40 2024-02-05T07:45:36.618 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:45:36.618 INFO:teuthology.orchestra.run.smithi131.stdout:[{"addr": "172.21.15.131", "hostname": "smithi131", "labels": [], "status": ""}, {"addr": "172.21.15.157", "hostname": "smithi157", "labels": [], "status": ""}] 2024-02-05T07:45:36.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.618+0000 7f40d49b8700 1 -- 172.21.15.131:0/3412626374 >> v2:172.21.15.131:6800/760045032 conn(0x7f40b8038f40 msgr2=0x7f40b803b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:36.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.618+0000 7f40d49b8700 1 --2- 172.21.15.131:0/3412626374 >> v2:172.21.15.131:6800/760045032 conn(0x7f40b8038f40 0x7f40b803b400 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f40c4006fd0 tx=0x7f40c4006e40 comp rx=0 tx=0).stop 2024-02-05T07:45:36.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.618+0000 7f40d49b8700 1 -- 172.21.15.131:0/3412626374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d01076d0 msgr2=0x7f40d0196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.618+0000 7f40d49b8700 1 --2- 172.21.15.131:0/3412626374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d01076d0 0x7f40d0196710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f40c000f690 tx=0x7f40c000f770 comp rx=0 tx=0).stop 2024-02-05T07:45:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.619+0000 7f40d49b8700 1 -- 172.21.15.131:0/3412626374 shutdown_connections 2024-02-05T07:45:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.619+0000 7f40d49b8700 1 --2- 172.21.15.131:0/3412626374 >> v2:172.21.15.131:6800/760045032 conn(0x7f40b8038f40 0x7f40b803b400 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.619+0000 7f40d49b8700 1 --2- 172.21.15.131:0/3412626374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40d01076d0 0x7f40d0196710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.619+0000 7f40d49b8700 1 -- 172.21.15.131:0/3412626374 >> 172.21.15.131:0/3412626374 conn(0x7f40d00fd200 msgr2=0x7f40d01094c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.619+0000 7f40d49b8700 1 -- 172.21.15.131:0/3412626374 shutdown_connections 2024-02-05T07:45:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:36.619+0000 7f40d49b8700 1 -- 172.21.15.131:0/3412626374 wait complete. 2024-02-05T07:45:37.243 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-05T07:45:37.244 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph osd crush tunables default 2024-02-05T07:45:37.476 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:45:37.775 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:37 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:37.775 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:37 smithi131 ceph-mon[29507]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:37.775 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:37 smithi131 ceph-mon[29507]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:45:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.069+0000 7f2d4eb75700 1 -- 172.21.15.131:0/2385364402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d48100f40 msgr2=0x7f2d48101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.069+0000 7f2d4eb75700 1 --2- 172.21.15.131:0/2385364402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d48100f40 0x7f2d48101320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009910 tx=0x7f2d3c009c20 comp rx=0 tx=0).stop 2024-02-05T07:45:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.069+0000 7f2d4eb75700 1 -- 172.21.15.131:0/2385364402 shutdown_connections 2024-02-05T07:45:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.069+0000 7f2d4eb75700 1 --2- 172.21.15.131:0/2385364402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d48100f40 0x7f2d48101320 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.069+0000 7f2d4eb75700 1 -- 172.21.15.131:0/2385364402 >> 172.21.15.131:0/2385364402 conn(0x7f2d480fb890 msgr2=0x7f2d480fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.070+0000 7f2d4eb75700 1 -- 172.21.15.131:0/2385364402 shutdown_connections 2024-02-05T07:45:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.070+0000 7f2d4eb75700 1 -- 172.21.15.131:0/2385364402 wait complete. 2024-02-05T07:45:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.070+0000 7f2d4eb75700 1 Processor -- start 2024-02-05T07:45:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.070+0000 7f2d4eb75700 1 -- start start 2024-02-05T07:45:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.070+0000 7f2d4eb75700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d48100f40 0x7f2d48197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.070+0000 7f2d4eb75700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d48100b80 con 0x7f2d48100f40 2024-02-05T07:45:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.070+0000 7f2d4c911700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d48100f40 0x7f2d48197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.070+0000 7f2d4c911700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d48100f40 0x7f2d48197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46618/0 (socket says 172.21.15.131:46618) 2024-02-05T07:45:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.070+0000 7f2d4c911700 1 -- 172.21.15.131:0/2130606218 learned_addr learned my addr 172.21.15.131:0/2130606218 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:45:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.071+0000 7f2d4c911700 1 -- 172.21.15.131:0/2130606218 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c0095f0 con 0x7f2d48100f40 2024-02-05T07:45:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.071+0000 7f2d4c911700 1 --2- 172.21.15.131:0/2130606218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d48100f40 0x7f2d48197590 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00f690 tx=0x7f2d3c00f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.071+0000 7f2d45ffb700 1 -- 172.21.15.131:0/2130606218 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d3c01d070 con 0x7f2d48100f40 2024-02-05T07:45:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.071+0000 7f2d45ffb700 1 -- 172.21.15.131:0/2130606218 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f2d3c004500 con 0x7f2d48100f40 2024-02-05T07:45:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.071+0000 7f2d4eb75700 1 -- 172.21.15.131:0/2130606218 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d48197b30 con 0x7f2d48100f40 2024-02-05T07:45:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.071+0000 7f2d45ffb700 1 -- 172.21.15.131:0/2130606218 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d3c003b50 con 0x7f2d48100f40 2024-02-05T07:45:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.071+0000 7f2d4eb75700 1 -- 172.21.15.131:0/2130606218 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d48197fb0 con 0x7f2d48100f40 2024-02-05T07:45:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.072+0000 7f2d45ffb700 1 -- 172.21.15.131:0/2130606218 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7f2d3c02b440 con 0x7f2d48100f40 2024-02-05T07:45:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.072+0000 7f2d4eb75700 1 -- 172.21.15.131:0/2130606218 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d481048c0 con 0x7f2d48100f40 2024-02-05T07:45:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.072+0000 7f2d45ffb700 1 --2- 172.21.15.131:0/2130606218 >> v2:172.21.15.131:6800/760045032 conn(0x7f2d38038f40 0x7f2d3803b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.072+0000 7f2d45ffb700 1 -- 172.21.15.131:0/2130606218 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f2d3c04cf20 con 0x7f2d48100f40 2024-02-05T07:45:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.072+0000 7f2d47fff700 1 --2- 172.21.15.131:0/2130606218 >> v2:172.21.15.131:6800/760045032 conn(0x7f2d38038f40 0x7f2d3803b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.073+0000 7f2d47fff700 1 --2- 172.21.15.131:0/2130606218 >> v2:172.21.15.131:6800/760045032 conn(0x7f2d38038f40 0x7f2d3803b400 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2d34006fd0 tx=0x7f2d34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:39.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.077+0000 7f2d45ffb700 1 -- 172.21.15.131:0/2130606218 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2d3c015720 con 0x7f2d48100f40 2024-02-05T07:45:39.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.276+0000 7f2d4eb75700 1 -- 172.21.15.131:0/2130606218 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f2d481001c0 con 0x7f2d48100f40 2024-02-05T07:45:39.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.674+0000 7f2d45ffb700 1 -- 172.21.15.131:0/2130606218 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f2d3c026070 con 0x7f2d48100f40 2024-02-05T07:45:39.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.677+0000 7f2d4eb75700 1 -- 172.21.15.131:0/2130606218 >> v2:172.21.15.131:6800/760045032 conn(0x7f2d38038f40 msgr2=0x7f2d3803b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:39.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.677+0000 7f2d4eb75700 1 --2- 172.21.15.131:0/2130606218 >> v2:172.21.15.131:6800/760045032 conn(0x7f2d38038f40 0x7f2d3803b400 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2d34006fd0 tx=0x7f2d34006e40 comp rx=0 tx=0).stop 2024-02-05T07:45:39.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.677+0000 7f2d4eb75700 1 -- 172.21.15.131:0/2130606218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d48100f40 msgr2=0x7f2d48197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:39.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.677+0000 7f2d4eb75700 1 --2- 172.21.15.131:0/2130606218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d48100f40 0x7f2d48197590 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00f690 tx=0x7f2d3c00f770 comp rx=0 tx=0).stop 2024-02-05T07:45:39.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.677+0000 7f2d4eb75700 1 -- 172.21.15.131:0/2130606218 shutdown_connections 2024-02-05T07:45:39.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.677+0000 7f2d4eb75700 1 --2- 172.21.15.131:0/2130606218 >> v2:172.21.15.131:6800/760045032 conn(0x7f2d38038f40 0x7f2d3803b400 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:39.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.677+0000 7f2d4eb75700 1 --2- 172.21.15.131:0/2130606218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d48100f40 0x7f2d48197590 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:39.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.677+0000 7f2d4eb75700 1 -- 172.21.15.131:0/2130606218 >> 172.21.15.131:0/2130606218 conn(0x7f2d480fb890 msgr2=0x7f2d480fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:39.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.677+0000 7f2d4eb75700 1 -- 172.21.15.131:0/2130606218 shutdown_connections 2024-02-05T07:45:39.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:45:39.677+0000 7f2d4eb75700 1 -- 172.21.15.131:0/2130606218 wait complete. 2024-02-05T07:45:39.680 INFO:teuthology.orchestra.run.smithi131.stderr:adjusted tunables profile to default 2024-02-05T07:45:39.998 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:39 smithi131 ceph-mon[29507]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:39.998 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2130606218' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-05T07:45:39.998 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:39 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:39.999 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:39 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:39.999 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:39 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:39.999 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:39 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:39.999 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:39 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:39.999 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:39 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:45:39.999 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:39 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:45:40.356 INFO:tasks.cephadm:Adding mon.a on smithi131 2024-02-05T07:45:40.356 INFO:tasks.cephadm:Adding mon.c on smithi131 2024-02-05T07:45:40.356 INFO:tasks.cephadm:Adding mon.b on smithi157 2024-02-05T07:45:40.356 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch apply mon '3;smithi131:172.21.15.131=a;smithi131:[v2:172.21.15.131:3301,v1:172.21.15.131:6790]=c;smithi157:172.21.15.157=b' 2024-02-05T07:45:40.588 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:45:41.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:40 smithi131 ceph-mon[29507]: Updating smithi157:/etc/ceph/ceph.conf 2024-02-05T07:45:41.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:40 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2130606218' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-05T07:45:41.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:40 smithi131 ceph-mon[29507]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T07:45:41.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:41.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:41.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:42.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:41 smithi131 ceph-mon[29507]: Updating smithi157:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:45:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:41 smithi131 ceph-mon[29507]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:45:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:41 smithi131 ceph-mon[29507]: Updating smithi157:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:45:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:41 smithi131 ceph-mon[29507]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:42.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.034+0000 7f32f6934700 1 -- 172.21.15.157:0/680252410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32f0107770 msgr2=0x7f32f0107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:42.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.034+0000 7f32f6934700 1 --2- 172.21.15.157:0/680252410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32f0107770 0x7f32f0107b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009910 tx=0x7f32e0009c20 comp rx=0 tx=0).stop 2024-02-05T07:45:42.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.034+0000 7f32f6934700 1 -- 172.21.15.157:0/680252410 shutdown_connections 2024-02-05T07:45:42.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.034+0000 7f32f6934700 1 --2- 172.21.15.157:0/680252410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32f0107770 0x7f32f0107b50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:42.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.034+0000 7f32f6934700 1 -- 172.21.15.157:0/680252410 >> 172.21.15.157:0/680252410 conn(0x7f32f00fb860 msgr2=0x7f32f00fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:42.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.034+0000 7f32f6934700 1 -- 172.21.15.157:0/680252410 shutdown_connections 2024-02-05T07:45:42.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.034+0000 7f32f6934700 1 -- 172.21.15.157:0/680252410 wait complete. 2024-02-05T07:45:42.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.035+0000 7f32f6934700 1 Processor -- start 2024-02-05T07:45:42.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.035+0000 7f32f6934700 1 -- start start 2024-02-05T07:45:42.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.035+0000 7f32f6934700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32f0107770 0x7f32f01966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:42.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.035+0000 7f32f6934700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f0196c00 con 0x7f32f0107770 2024-02-05T07:45:42.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.035+0000 7f32effff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32f0107770 0x7f32f01966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:42.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.035+0000 7f32effff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32f0107770 0x7f32f01966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.157:55348/0 (socket says 172.21.15.157:55348) 2024-02-05T07:45:42.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.035+0000 7f32effff700 1 -- 172.21.15.157:0/4028560407 learned_addr learned my addr 172.21.15.157:0/4028560407 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:45:42.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.035+0000 7f32effff700 1 -- 172.21.15.157:0/4028560407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e00095f0 con 0x7f32f0107770 2024-02-05T07:45:42.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.036+0000 7f32effff700 1 --2- 172.21.15.157:0/4028560407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32f0107770 0x7f32f01966c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f32e000f690 tx=0x7f32e000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:42.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.036+0000 7f32ed7fa700 1 -- 172.21.15.157:0/4028560407 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32e001d070 con 0x7f32f0107770 2024-02-05T07:45:42.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.036+0000 7f32ed7fa700 1 -- 172.21.15.157:0/4028560407 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f32e0004500 con 0x7f32f0107770 2024-02-05T07:45:42.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.036+0000 7f32f6934700 1 -- 172.21.15.157:0/4028560407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f0196e60 con 0x7f32f0107770 2024-02-05T07:45:42.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.036+0000 7f32ed7fa700 1 -- 172.21.15.157:0/4028560407 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32e0003b30 con 0x7f32f0107770 2024-02-05T07:45:42.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.036+0000 7f32f6934700 1 -- 172.21.15.157:0/4028560407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f0197360 con 0x7f32f0107770 2024-02-05T07:45:42.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.037+0000 7f32ed7fa700 1 -- 172.21.15.157:0/4028560407 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7f32e0004050 con 0x7f32f0107770 2024-02-05T07:45:42.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.037+0000 7f32f6934700 1 -- 172.21.15.157:0/4028560407 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f0063350 con 0x7f32f0107770 2024-02-05T07:45:42.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.037+0000 7f32ed7fa700 1 --2- 172.21.15.157:0/4028560407 >> v2:172.21.15.131:6800/760045032 conn(0x7f32d8038f40 0x7f32d803b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:42.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.037+0000 7f32ed7fa700 1 -- 172.21.15.157:0/4028560407 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f32e004cb20 con 0x7f32f0107770 2024-02-05T07:45:42.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.037+0000 7f32ef7fe700 1 --2- 172.21.15.157:0/4028560407 >> v2:172.21.15.131:6800/760045032 conn(0x7f32d8038f40 0x7f32d803b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:42.039 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.038+0000 7f32ef7fe700 1 --2- 172.21.15.157:0/4028560407 >> v2:172.21.15.131:6800/760045032 conn(0x7f32d8038f40 0x7f32d803b400 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f32e4006fd0 tx=0x7f32e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:42.042 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.042+0000 7f32ed7fa700 1 -- 172.21.15.157:0/4028560407 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f32e0015350 con 0x7f32f0107770 2024-02-05T07:45:42.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.246+0000 7f32f6934700 1 -- 172.21.15.157:0/4028560407 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:172.21.15.131=a;smithi131:[v2:172.21.15.131:3301,v1:172.21.15.131:6790]=c;smithi157:172.21.15.157=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f32f0197360 con 0x7f32d8038f40 2024-02-05T07:45:42.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.252+0000 7f32ed7fa700 1 -- 172.21.15.157:0/4028560407 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f32f0197360 con 0x7f32d8038f40 2024-02-05T07:45:42.253 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mon update... 2024-02-05T07:45:42.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.254+0000 7f32f6934700 1 -- 172.21.15.157:0/4028560407 >> v2:172.21.15.131:6800/760045032 conn(0x7f32d8038f40 msgr2=0x7f32d803b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:42.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.254+0000 7f32f6934700 1 --2- 172.21.15.157:0/4028560407 >> v2:172.21.15.131:6800/760045032 conn(0x7f32d8038f40 0x7f32d803b400 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f32e4006fd0 tx=0x7f32e4006e40 comp rx=0 tx=0).stop 2024-02-05T07:45:42.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.254+0000 7f32f6934700 1 -- 172.21.15.157:0/4028560407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32f0107770 msgr2=0x7f32f01966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:42.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.254+0000 7f32f6934700 1 --2- 172.21.15.157:0/4028560407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32f0107770 0x7f32f01966c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f32e000f690 tx=0x7f32e000f770 comp rx=0 tx=0).stop 2024-02-05T07:45:42.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.255+0000 7f32f6934700 1 -- 172.21.15.157:0/4028560407 shutdown_connections 2024-02-05T07:45:42.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.255+0000 7f32f6934700 1 --2- 172.21.15.157:0/4028560407 >> v2:172.21.15.131:6800/760045032 conn(0x7f32d8038f40 0x7f32d803b400 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:42.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.255+0000 7f32f6934700 1 --2- 172.21.15.157:0/4028560407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32f0107770 0x7f32f01966c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:42.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.255+0000 7f32f6934700 1 -- 172.21.15.157:0/4028560407 >> 172.21.15.157:0/4028560407 conn(0x7f32f00fb860 msgr2=0x7f32f00fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:42.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.255+0000 7f32f6934700 1 -- 172.21.15.157:0/4028560407 shutdown_connections 2024-02-05T07:45:42.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:42.255+0000 7f32f6934700 1 -- 172.21.15.157:0/4028560407 wait complete. 2024-02-05T07:45:42.850 DEBUG:teuthology.orchestra.run.smithi131:mon.c> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mon.c.service 2024-02-05T07:45:42.853 DEBUG:teuthology.orchestra.run.smithi157:mon.b> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mon.b.service 2024-02-05T07:45:42.856 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-05T07:45:42.856 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-05T07:45:43.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:43 smithi131 ceph-mon[29507]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:172.21.15.131=a;smithi131:[v2:172.21.15.131:3301,v1:172.21.15.131:6790]=c;smithi157:172.21.15.157=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:45:43.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:43 smithi131 ceph-mon[29507]: Saving service mon spec with placement smithi131:172.21.15.131=a;smithi131:[v2:172.21.15.131:3301,v1:172.21.15.131:6790]=c;smithi157:172.21.15.157=b;count:3 2024-02-05T07:45:43.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:43.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:45:43.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:45:43.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:45:43.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:43.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:45:43.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:45:43.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:43 smithi131 ceph-mon[29507]: Deploying daemon mon.b on smithi157 2024-02-05T07:45:43.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:43 smithi131 ceph-mon[29507]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:43.582 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:45:46.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:45 smithi131 ceph-mon[29507]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:46.847 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.844+0000 7f88d00a2700 1 -- 172.21.15.157:0/2240906491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88c8106690 msgr2=0x7f88c8106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:46.847 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.844+0000 7f88d00a2700 1 --2- 172.21.15.157:0/2240906491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88c8106690 0x7f88c8106a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009910 tx=0x7f88b8009c20 comp rx=0 tx=0).stop 2024-02-05T07:45:46.847 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.844+0000 7f88d00a2700 1 -- 172.21.15.157:0/2240906491 shutdown_connections 2024-02-05T07:45:46.847 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.844+0000 7f88d00a2700 1 --2- 172.21.15.157:0/2240906491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88c8106690 0x7f88c8106a70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:46.847 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.844+0000 7f88d00a2700 1 -- 172.21.15.157:0/2240906491 >> 172.21.15.157:0/2240906491 conn(0x7f88c80fc130 msgr2=0x7f88c80fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:46.847 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.845+0000 7f88d00a2700 1 -- 172.21.15.157:0/2240906491 shutdown_connections 2024-02-05T07:45:46.847 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.845+0000 7f88d00a2700 1 -- 172.21.15.157:0/2240906491 wait complete. 2024-02-05T07:45:46.848 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.845+0000 7f88d00a2700 1 Processor -- start 2024-02-05T07:45:46.848 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.845+0000 7f88d00a2700 1 -- start start 2024-02-05T07:45:46.848 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.845+0000 7f88d00a2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88c8106690 0x7f88c8197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:46.848 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.845+0000 7f88d00a2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c806aae0 con 0x7f88c8106690 2024-02-05T07:45:46.848 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.846+0000 7f88cde3e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88c8106690 0x7f88c8197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:46.848 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.846+0000 7f88cde3e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88c8106690 0x7f88c8197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.157:41816/0 (socket says 172.21.15.157:41816) 2024-02-05T07:45:46.848 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.846+0000 7f88cde3e700 1 -- 172.21.15.157:0/3355490059 learned_addr learned my addr 172.21.15.157:0/3355490059 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:45:46.848 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.846+0000 7f88cde3e700 1 -- 172.21.15.157:0/3355490059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b80095f0 con 0x7f88c8106690 2024-02-05T07:45:46.848 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.846+0000 7f88cde3e700 1 --2- 172.21.15.157:0/3355490059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88c8106690 0x7f88c8197520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f88b80092e0 tx=0x7f88b800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:46.848 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.846+0000 7f88beffd700 1 -- 172.21.15.157:0/3355490059 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88b801d070 con 0x7f88c8106690 2024-02-05T07:45:46.848 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.846+0000 7f88beffd700 1 -- 172.21.15.157:0/3355490059 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f88b8004500 con 0x7f88c8106690 2024-02-05T07:45:46.848 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.846+0000 7f88d00a2700 1 -- 172.21.15.157:0/3355490059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c8197b20 con 0x7f88c8106690 2024-02-05T07:45:46.848 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.846+0000 7f88beffd700 1 -- 172.21.15.157:0/3355490059 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88b8003b80 con 0x7f88c8106690 2024-02-05T07:45:46.848 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.846+0000 7f88d00a2700 1 -- 172.21.15.157:0/3355490059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c8197fa0 con 0x7f88c8106690 2024-02-05T07:45:46.849 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.847+0000 7f88beffd700 1 -- 172.21.15.157:0/3355490059 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7f88b802b440 con 0x7f88c8106690 2024-02-05T07:45:46.849 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.847+0000 7f88d00a2700 1 -- 172.21.15.157:0/3355490059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c8105980 con 0x7f88c8106690 2024-02-05T07:45:46.849 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.847+0000 7f88beffd700 1 --2- 172.21.15.157:0/3355490059 >> v2:172.21.15.131:6800/760045032 conn(0x7f88b4038f40 0x7f88b403b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:46.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.847+0000 7f88beffd700 1 -- 172.21.15.157:0/3355490059 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f88b804cd60 con 0x7f88c8106690 2024-02-05T07:45:46.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.847+0000 7f88cd63d700 1 --2- 172.21.15.157:0/3355490059 >> v2:172.21.15.131:6800/760045032 conn(0x7f88b4038f40 0x7f88b403b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:46.850 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.848+0000 7f88cd63d700 1 --2- 172.21.15.157:0/3355490059 >> v2:172.21.15.131:6800/760045032 conn(0x7f88b4038f40 0x7f88b403b400 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f88c4006fd0 tx=0x7f88c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:46.853 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:46.852+0000 7f88beffd700 1 -- 172.21.15.157:0/3355490059 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f88b8015760 con 0x7f88c8106690 2024-02-05T07:45:47.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:47.135+0000 7f88d00a2700 1 -- 172.21.15.157:0/3355490059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f88c81981f0 con 0x7f88c8106690 2024-02-05T07:45:47.136 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:47.135+0000 7f88beffd700 1 -- 172.21.15.157:0/3355490059 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+749 (secure 0 0 0) 0x7f88b8026070 con 0x7f88c8106690 2024-02-05T07:45:47.137 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:45:47.137 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"4018fbe8-c3fa-11ee-95b6-87774f69a715","modified":"2024-02-05T07:44:04.773825Z","created":"2024-02-05T07:44:04.773825Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-05T07:45:47.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:47.138+0000 7f88d00a2700 1 -- 172.21.15.157:0/3355490059 >> v2:172.21.15.131:6800/760045032 conn(0x7f88b4038f40 msgr2=0x7f88b403b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:47.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:47.138+0000 7f88d00a2700 1 --2- 172.21.15.157:0/3355490059 >> v2:172.21.15.131:6800/760045032 conn(0x7f88b4038f40 0x7f88b403b400 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f88c4006fd0 tx=0x7f88c4006e40 comp rx=0 tx=0).stop 2024-02-05T07:45:47.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:47.138+0000 7f88d00a2700 1 -- 172.21.15.157:0/3355490059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88c8106690 msgr2=0x7f88c8197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:47.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:47.138+0000 7f88d00a2700 1 --2- 172.21.15.157:0/3355490059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88c8106690 0x7f88c8197520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f88b80092e0 tx=0x7f88b800f740 comp rx=0 tx=0).stop 2024-02-05T07:45:47.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:47.138+0000 7f88d00a2700 1 -- 172.21.15.157:0/3355490059 shutdown_connections 2024-02-05T07:45:47.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:47.138+0000 7f88d00a2700 1 --2- 172.21.15.157:0/3355490059 >> v2:172.21.15.131:6800/760045032 conn(0x7f88b4038f40 0x7f88b403b400 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:47.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:47.138+0000 7f88d00a2700 1 --2- 172.21.15.157:0/3355490059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88c8106690 0x7f88c8197520 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:47.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:47.138+0000 7f88d00a2700 1 -- 172.21.15.157:0/3355490059 >> 172.21.15.157:0/3355490059 conn(0x7f88c80fc130 msgr2=0x7f88c80fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:47.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:47.138+0000 7f88d00a2700 1 -- 172.21.15.157:0/3355490059 shutdown_connections 2024-02-05T07:45:47.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:47.138+0000 7f88d00a2700 1 -- 172.21.15.157:0/3355490059 wait complete. 2024-02-05T07:45:47.139 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-02-05T07:45:49.036 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-05T07:45:49.036 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-05T07:45:49.297 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:45:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:45:52 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:45:53.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:45:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:45:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: mon.a calling monitor election 2024-02-05T07:45:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:45:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:45:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: mon.b calling monitor election 2024-02-05T07:45:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:45:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:45:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:45:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-05T07:45:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: monmap e2: 2 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T07:45:53.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: fsmap 2024-02-05T07:45:53.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T07:45:53.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: mgrmap e13: y(active, since 41s) 2024-02-05T07:45:53.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: overall HEALTH_OK 2024-02-05T07:45:53.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:53.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:53.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:53.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:45:53.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:52 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:45:53.685 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.683+0000 7f38f77ba700 1 -- 172.21.15.157:0/159975738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc0035c0 msgr2=0x7f38dc005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:53.686 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.683+0000 7f38f77ba700 1 --2- 172.21.15.157:0/159975738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc0035c0 0x7f38dc005a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f38f0078010 tx=0x7f38ec00feb0 comp rx=0 tx=0).stop 2024-02-05T07:45:53.686 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.684+0000 7f38f77ba700 1 -- 172.21.15.157:0/159975738 shutdown_connections 2024-02-05T07:45:53.686 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.684+0000 7f38f77ba700 1 --2- 172.21.15.157:0/159975738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc0035c0 0x7f38dc005a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:53.686 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.684+0000 7f38f77ba700 1 -- 172.21.15.157:0/159975738 >> 172.21.15.157:0/159975738 conn(0x7f38f0076dd0 msgr2=0x7f38f00771e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:53.686 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.684+0000 7f38f77ba700 1 -- 172.21.15.157:0/159975738 shutdown_connections 2024-02-05T07:45:53.686 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.684+0000 7f38f77ba700 1 -- 172.21.15.157:0/159975738 wait complete. 2024-02-05T07:45:53.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.685+0000 7f38f77ba700 1 Processor -- start 2024-02-05T07:45:53.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.685+0000 7f38f77ba700 1 -- start start 2024-02-05T07:45:53.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.685+0000 7f38f77ba700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc0035c0 0x7f38f01a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:53.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.685+0000 7f38f77ba700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f01a2050 0x7f38f019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:53.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.685+0000 7f38f77ba700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f01a2570 con 0x7f38dc0035c0 2024-02-05T07:45:53.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.685+0000 7f38f77ba700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f01a26e0 con 0x7f38f01a2050 2024-02-05T07:45:53.688 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.685+0000 7f38f5556700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc0035c0 0x7f38f01a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:53.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.685+0000 7f38f4d55700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f01a2050 0x7f38f019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:53.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.685+0000 7f38f5556700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc0035c0 0x7f38f01a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.157:41868/0 (socket says 172.21.15.157:41868) 2024-02-05T07:45:53.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.685+0000 7f38f5556700 1 -- 172.21.15.157:0/642683668 learned_addr learned my addr 172.21.15.157:0/642683668 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:45:53.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.686+0000 7f38f4d55700 1 -- 172.21.15.157:0/642683668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f01a2050 msgr2=0x7f38f019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-05T07:45:53.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.686+0000 7f38f4d55700 1 -- 172.21.15.157:0/642683668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f01a2050 msgr2=0x7f38f019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-05T07:45:53.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.686+0000 7f38f4d55700 1 --2- 172.21.15.157:0/642683668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f01a2050 0x7f38f019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-05T07:45:53.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.686+0000 7f38f4d55700 1 --2- 172.21.15.157:0/642683668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f01a2050 0x7f38f019ce70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-05T07:45:53.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.686+0000 7f38f5556700 1 -- 172.21.15.157:0/642683668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f01a2050 msgr2=0x7f38f019ce70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-05T07:45:53.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.686+0000 7f38f5556700 1 --2- 172.21.15.157:0/642683668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f01a2050 0x7f38f019ce70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:53.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.686+0000 7f38f5556700 1 -- 172.21.15.157:0/642683668 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec00f900 con 0x7f38dc0035c0 2024-02-05T07:45:53.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.686+0000 7f38f5556700 1 --2- 172.21.15.157:0/642683668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc0035c0 0x7f38f01a1b10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0057a0 tx=0x7f38ec004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:53.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.686+0000 7f38e67fc700 1 -- 172.21.15.157:0/642683668 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f38ec013b00 con 0x7f38dc0035c0 2024-02-05T07:45:53.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.686+0000 7f38e67fc700 1 -- 172.21.15.157:0/642683668 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f38ec004b60 con 0x7f38dc0035c0 2024-02-05T07:45:53.689 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.686+0000 7f38e67fc700 1 -- 172.21.15.157:0/642683668 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f38ec003c20 con 0x7f38dc0035c0 2024-02-05T07:45:53.690 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.686+0000 7f38f77ba700 1 -- 172.21.15.157:0/642683668 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f019d410 con 0x7f38dc0035c0 2024-02-05T07:45:53.690 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.686+0000 7f38f77ba700 1 -- 172.21.15.157:0/642683668 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f019d960 con 0x7f38dc0035c0 2024-02-05T07:45:53.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.687+0000 7f38e67fc700 1 -- 172.21.15.157:0/642683668 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7f38ec013cf0 con 0x7f38dc0035c0 2024-02-05T07:45:53.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.687+0000 7f38f77ba700 1 -- 172.21.15.157:0/642683668 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f0196c80 con 0x7f38dc0035c0 2024-02-05T07:45:53.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.687+0000 7f38e67fc700 1 --2- 172.21.15.157:0/642683668 >> v2:172.21.15.131:6800/760045032 conn(0x7f38d4038d30 0x7f38d403b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:45:53.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.688+0000 7f38e67fc700 1 -- 172.21.15.157:0/642683668 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f38ec04e5f0 con 0x7f38dc0035c0 2024-02-05T07:45:53.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.688+0000 7f38f4d55700 1 --2- 172.21.15.157:0/642683668 >> v2:172.21.15.131:6800/760045032 conn(0x7f38d4038d30 0x7f38d403b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:45:53.691 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.688+0000 7f38f4d55700 1 --2- 172.21.15.157:0/642683668 >> v2:172.21.15.131:6800/760045032 conn(0x7f38d4038d30 0x7f38d403b1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009850 tx=0x7f38e0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:45:53.694 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.693+0000 7f38e67fc700 1 -- 172.21.15.157:0/642683668 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f38ec034530 con 0x7f38dc0035c0 2024-02-05T07:45:53.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:53 smithi131 ceph-mon[29507]: Deploying daemon mon.c on smithi131 2024-02-05T07:45:53.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:53 smithi131 ceph-mon[29507]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:53.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:53 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:45:53.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.966+0000 7f38f77ba700 1 -- 172.21.15.157:0/642683668 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f38f019e330 con 0x7f38dc0035c0 2024-02-05T07:45:53.968 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:45:53 smithi157 ceph-mon[33345]: Deploying daemon mon.c on smithi131 2024-02-05T07:45:53.968 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:45:53 smithi157 ceph-mon[33345]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:53.968 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:45:53 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:45:53.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.966+0000 7f38e67fc700 1 -- 172.21.15.157:0/642683668 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1019 (secure 0 0 0) 0x7f38ec034350 con 0x7f38dc0035c0 2024-02-05T07:45:53.969 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:45:53.969 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":2,"fsid":"4018fbe8-c3fa-11ee-95b6-87774f69a715","modified":"2024-02-05T07:45:47.465443Z","created":"2024-02-05T07:44:04.773825Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-05T07:45:53.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.969+0000 7f38f77ba700 1 -- 172.21.15.157:0/642683668 >> v2:172.21.15.131:6800/760045032 conn(0x7f38d4038d30 msgr2=0x7f38d403b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:53.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.969+0000 7f38f77ba700 1 --2- 172.21.15.157:0/642683668 >> v2:172.21.15.131:6800/760045032 conn(0x7f38d4038d30 0x7f38d403b1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009850 tx=0x7f38e0008040 comp rx=0 tx=0).stop 2024-02-05T07:45:53.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.969+0000 7f38f77ba700 1 -- 172.21.15.157:0/642683668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc0035c0 msgr2=0x7f38f01a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:45:53.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.969+0000 7f38f77ba700 1 --2- 172.21.15.157:0/642683668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc0035c0 0x7f38f01a1b10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0057a0 tx=0x7f38ec004f40 comp rx=0 tx=0).stop 2024-02-05T07:45:53.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.969+0000 7f38f77ba700 1 -- 172.21.15.157:0/642683668 shutdown_connections 2024-02-05T07:45:53.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.969+0000 7f38f77ba700 1 --2- 172.21.15.157:0/642683668 >> v2:172.21.15.131:6800/760045032 conn(0x7f38d4038d30 0x7f38d403b1f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:53.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.969+0000 7f38f77ba700 1 --2- 172.21.15.157:0/642683668 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc0035c0 0x7f38f01a1b10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:53.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.969+0000 7f38f77ba700 1 --2- 172.21.15.157:0/642683668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38f01a2050 0x7f38f019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:45:53.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.969+0000 7f38f77ba700 1 -- 172.21.15.157:0/642683668 >> 172.21.15.157:0/642683668 conn(0x7f38f0076dd0 msgr2=0x7f38f00fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:45:53.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.969+0000 7f38f77ba700 1 -- 172.21.15.157:0/642683668 shutdown_connections 2024-02-05T07:45:53.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:45:53.969+0000 7f38f77ba700 1 -- 172.21.15.157:0/642683668 wait complete. 2024-02-05T07:45:53.972 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 2 2024-02-05T07:45:54.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.157:0/642683668' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T07:45:54.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:45:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.157:0/642683668' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T07:45:55.436 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-05T07:45:55.436 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-05T07:45:55.679 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:55 smithi131 systemd[1]: Starting Ceph mon.c for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T07:45:55.680 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:45:55.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:45:55 smithi157 ceph-mon[33345]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:56.005 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:45:55 smithi131 ceph-mon[29507]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:56.005 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:55 smithi131 podman[41616]: 2024-02-05 07:45:55.683430218 +0000 UTC m=+0.018500144 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T07:45:56.005 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:55 smithi131 podman[41616]: 2024-02-05 07:45:55.891029386 +0000 UTC m=+0.226099313 container create 72bec76edf25c9a73252ee7790d77c8c2e8eafe79d34854d2b931bf13ec48a24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux ) 2024-02-05T07:45:56.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 podman[41616]: 2024-02-05 07:45:56.099553297 +0000 UTC m=+0.434623224 container init 72bec76edf25c9a73252ee7790d77c8c2e8eafe79d34854d2b931bf13ec48a24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-c, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T07:45:56.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 podman[41616]: 2024-02-05 07:45:56.105504303 +0000 UTC m=+0.440574231 container start 72bec76edf25c9a73252ee7790d77c8c2e8eafe79d34854d2b931bf13ec48a24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T07:45:56.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: set uid:gid to 167:167 (ceph:ceph) 2024-02-05T07:45:56.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev), process ceph-mon, pid 7 2024-02-05T07:45:56.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: pidfile_write: ignore empty --pid-file 2024-02-05T07:45:56.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: load: jerasure load: lrc 2024-02-05T07:45:56.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: RocksDB version: 7.9.2 2024-02-05T07:45:56.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Git sha 0 2024-02-05T07:45:56.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Compile date 2024-02-04 23:11:06 2024-02-05T07:45:56.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: DB SUMMARY 2024-02-05T07:45:56.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: DB Session ID: 03BOYWKNQXY5T4JHAMG3 2024-02-05T07:45:56.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: CURRENT file: CURRENT 2024-02-05T07:45:56.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: IDENTITY file: IDENTITY 2024-02-05T07:45:56.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-05T07:45:56.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-05T07:45:56.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-02-05T07:45:56.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.error_if_exists: 0 2024-02-05T07:45:56.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.create_if_missing: 0 2024-02-05T07:45:56.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.paranoid_checks: 1 2024-02-05T07:45:56.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-05T07:45:56.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-05T07:45:56.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-05T07:45:56.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.env: 0x557f3a703ae0 2024-02-05T07:45:56.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.fs: PosixFileSystem 2024-02-05T07:45:56.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.info_log: 0x557f3cb1b8c0 2024-02-05T07:45:56.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_file_opening_threads: 16 2024-02-05T07:45:56.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.statistics: (nil) 2024-02-05T07:45:56.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.use_fsync: 0 2024-02-05T07:45:56.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_log_file_size: 0 2024-02-05T07:45:56.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-05T07:45:56.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-05T07:45:56.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.keep_log_file_num: 1000 2024-02-05T07:45:56.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.recycle_log_file_num: 0 2024-02-05T07:45:56.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.allow_fallocate: 1 2024-02-05T07:45:56.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.allow_mmap_reads: 0 2024-02-05T07:45:56.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.allow_mmap_writes: 0 2024-02-05T07:45:56.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.use_direct_reads: 0 2024-02-05T07:45:56.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-05T07:45:56.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.create_missing_column_families: 0 2024-02-05T07:45:56.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.db_log_dir: 2024-02-05T07:45:56.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.wal_dir: 2024-02-05T07:45:56.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-05T07:45:56.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-05T07:45:56.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-05T07:45:56.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-05T07:45:56.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-05T07:45:56.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-05T07:45:56.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.advise_random_on_open: 1 2024-02-05T07:45:56.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.db_write_buffer_size: 0 2024-02-05T07:45:56.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.write_buffer_manager: 0x557f3caef400 2024-02-05T07:45:56.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-05T07:45:56.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-05T07:45:56.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-05T07:45:56.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.rate_limiter: (nil) 2024-02-05T07:45:56.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-05T07:45:56.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.wal_recovery_mode: 2 2024-02-05T07:45:56.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.enable_thread_tracking: 0 2024-02-05T07:45:56.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.enable_pipelined_write: 0 2024-02-05T07:45:56.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.unordered_write: 0 2024-02-05T07:45:56.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-05T07:45:56.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-05T07:45:56.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-05T07:45:56.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-05T07:45:56.524 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.row_cache: None 2024-02-05T07:45:56.524 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.wal_filter: None 2024-02-05T07:45:56.524 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-05T07:45:56.524 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.allow_ingest_behind: 0 2024-02-05T07:45:56.524 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.two_write_queues: 0 2024-02-05T07:45:56.524 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.manual_wal_flush: 0 2024-02-05T07:45:56.524 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.wal_compression: 0 2024-02-05T07:45:56.524 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.atomic_flush: 0 2024-02-05T07:45:56.524 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-05T07:45:56.524 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-05T07:45:56.524 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-05T07:45:56.524 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.log_readahead_size: 0 2024-02-05T07:45:56.524 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-05T07:45:56.524 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.best_efforts_recovery: 0 2024-02-05T07:45:56.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-05T07:45:56.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-05T07:45:56.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.allow_data_in_errors: 0 2024-02-05T07:45:56.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.db_host_id: __hostname__ 2024-02-05T07:45:56.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-05T07:45:56.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_background_jobs: 2 2024-02-05T07:45:56.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_background_compactions: -1 2024-02-05T07:45:56.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_subcompactions: 1 2024-02-05T07:45:56.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-05T07:45:56.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-05T07:45:56.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-05T07:45:56.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_total_wal_size: 0 2024-02-05T07:45:56.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-05T07:45:56.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-05T07:45:56.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-05T07:45:56.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-05T07:45:56.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_open_files: -1 2024-02-05T07:45:56.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.bytes_per_sync: 0 2024-02-05T07:45:56.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-05T07:45:56.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-05T07:45:56.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compaction_readahead_size: 0 2024-02-05T07:45:56.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_background_flushes: -1 2024-02-05T07:45:56.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Compression algorithms supported: 2024-02-05T07:45:56.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-05T07:45:56.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: kZSTD supported: 0 2024-02-05T07:45:56.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: kXpressCompression supported: 0 2024-02-05T07:45:56.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: kLZ4HCCompression supported: 1 2024-02-05T07:45:56.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: kZlibCompression supported: 1 2024-02-05T07:45:56.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: kSnappyCompression supported: 1 2024-02-05T07:45:56.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: kLZ4Compression supported: 1 2024-02-05T07:45:56.527 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: kBZip2Compression supported: 0 2024-02-05T07:45:56.527 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-05T07:45:56.527 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-05T07:45:56.527 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-02-05T07:45:56.527 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-05T07:45:56.527 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-05T07:45:56.527 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.merge_operator: 2024-02-05T07:45:56.527 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compaction_filter: None 2024-02-05T07:45:56.527 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compaction_filter_factory: None 2024-02-05T07:45:56.527 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.sst_partitioner_factory: None 2024-02-05T07:45:56.527 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-05T07:45:56.527 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-05T07:45:56.527 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557f3cb1b9c0) 2024-02-05T07:45:56.527 INFO:journalctl@ceph.mon.c.smithi131.stdout: cache_index_and_filter_blocks: 1 2024-02-05T07:45:56.528 INFO:journalctl@ceph.mon.c.smithi131.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-05T07:45:56.528 INFO:journalctl@ceph.mon.c.smithi131.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-05T07:45:56.528 INFO:journalctl@ceph.mon.c.smithi131.stdout: pin_top_level_index_and_filter: 1 2024-02-05T07:45:56.528 INFO:journalctl@ceph.mon.c.smithi131.stdout: index_type: 0 2024-02-05T07:45:56.528 INFO:journalctl@ceph.mon.c.smithi131.stdout: data_block_index_type: 0 2024-02-05T07:45:56.528 INFO:journalctl@ceph.mon.c.smithi131.stdout: index_shortening: 1 2024-02-05T07:45:56.528 INFO:journalctl@ceph.mon.c.smithi131.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-05T07:45:56.528 INFO:journalctl@ceph.mon.c.smithi131.stdout: checksum: 4 2024-02-05T07:45:56.528 INFO:journalctl@ceph.mon.c.smithi131.stdout: no_block_cache: 0 2024-02-05T07:45:56.528 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache: 0x557f3cb1f090 2024-02-05T07:45:56.528 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache_name: BinnedLRUCache 2024-02-05T07:45:56.528 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache_options: 2024-02-05T07:45:56.528 INFO:journalctl@ceph.mon.c.smithi131.stdout: capacity : 536870912 2024-02-05T07:45:56.528 INFO:journalctl@ceph.mon.c.smithi131.stdout: num_shard_bits : 4 2024-02-05T07:45:56.529 INFO:journalctl@ceph.mon.c.smithi131.stdout: strict_capacity_limit : 0 2024-02-05T07:45:56.529 INFO:journalctl@ceph.mon.c.smithi131.stdout: high_pri_pool_ratio: 0.000 2024-02-05T07:45:56.529 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache_compressed: (nil) 2024-02-05T07:45:56.529 INFO:journalctl@ceph.mon.c.smithi131.stdout: persistent_cache: (nil) 2024-02-05T07:45:56.529 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_size: 4096 2024-02-05T07:45:56.529 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_size_deviation: 10 2024-02-05T07:45:56.529 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_restart_interval: 16 2024-02-05T07:45:56.529 INFO:journalctl@ceph.mon.c.smithi131.stdout: index_block_restart_interval: 1 2024-02-05T07:45:56.529 INFO:journalctl@ceph.mon.c.smithi131.stdout: metadata_block_size: 4096 2024-02-05T07:45:56.529 INFO:journalctl@ceph.mon.c.smithi131.stdout: partition_filters: 0 2024-02-05T07:45:56.529 INFO:journalctl@ceph.mon.c.smithi131.stdout: use_delta_encoding: 1 2024-02-05T07:45:56.529 INFO:journalctl@ceph.mon.c.smithi131.stdout: filter_policy: bloomfilter 2024-02-05T07:45:56.529 INFO:journalctl@ceph.mon.c.smithi131.stdout: whole_key_filtering: 1 2024-02-05T07:45:56.529 INFO:journalctl@ceph.mon.c.smithi131.stdout: verify_compression: 0 2024-02-05T07:45:56.529 INFO:journalctl@ceph.mon.c.smithi131.stdout: read_amp_bytes_per_bit: 0 2024-02-05T07:45:56.529 INFO:journalctl@ceph.mon.c.smithi131.stdout: format_version: 5 2024-02-05T07:45:56.530 INFO:journalctl@ceph.mon.c.smithi131.stdout: enable_index_compression: 1 2024-02-05T07:45:56.530 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_align: 0 2024-02-05T07:45:56.530 INFO:journalctl@ceph.mon.c.smithi131.stdout: max_auto_readahead_size: 262144 2024-02-05T07:45:56.530 INFO:journalctl@ceph.mon.c.smithi131.stdout: prepopulate_block_cache: 0 2024-02-05T07:45:56.530 INFO:journalctl@ceph.mon.c.smithi131.stdout: initial_auto_readahead_size: 8192 2024-02-05T07:45:56.530 INFO:journalctl@ceph.mon.c.smithi131.stdout: num_file_reads_for_auto_readahead: 2 2024-02-05T07:45:56.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.write_buffer_size: 33554432 2024-02-05T07:45:56.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_write_buffer_number: 2 2024-02-05T07:45:56.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compression: NoCompression 2024-02-05T07:45:56.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.bottommost_compression: Disabled 2024-02-05T07:45:56.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.prefix_extractor: nullptr 2024-02-05T07:45:56.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-05T07:45:56.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.num_levels: 7 2024-02-05T07:45:56.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-05T07:45:56.530 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compression_opts.level: 32767 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compression_opts.strategy: 0 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-05T07:45:56.531 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compression_opts.enabled: false 2024-02-05T07:45:56.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-05T07:45:56.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-05T07:45:56.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-05T07:45:56.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-05T07:45:56.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.target_file_size_base: 67108864 2024-02-05T07:45:56.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-05T07:45:56.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-05T07:45:56.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-05T07:45:56.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-05T07:45:56.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-05T07:45:56.532 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-05T07:45:56.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-05T07:45:56.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-05T07:45:56.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-05T07:45:56.533 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-05T07:45:56.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-05T07:45:56.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-05T07:45:56.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-05T07:45:56.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-05T07:45:56.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.arena_block_size: 1048576 2024-02-05T07:45:56.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-05T07:45:56.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-05T07:45:56.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.disable_auto_compactions: 0 2024-02-05T07:45:56.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-05T07:45:56.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-05T07:45:56.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-05T07:45:56.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-05T07:45:56.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-05T07:45:56.534 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-05T07:45:56.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-05T07:45:56.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-05T07:45:56.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-05T07:45:56.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-05T07:45:56.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.table_properties_collectors: 2024-02-05T07:45:56.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.inplace_update_support: 0 2024-02-05T07:45:56.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-05T07:45:56.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-05T07:45:56.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-05T07:45:56.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-05T07:45:56.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.bloom_locality: 0 2024-02-05T07:45:56.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.max_successive_merges: 0 2024-02-05T07:45:56.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-05T07:45:56.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.paranoid_file_checks: 0 2024-02-05T07:45:56.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.force_consistency_checks: 1 2024-02-05T07:45:56.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.report_bg_io_stats: 0 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.ttl: 2592000 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.enable_blob_files: false 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.min_blob_size: 0 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.blob_file_size: 268435456 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.blob_file_starting_level: 0 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-05T07:45:56.536 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 63d6440e-357a-4647-9b53-41365371aff9 2024-02-05T07:45:56.537 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707119156135601, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-05T07:45:56.537 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-05T07:45:56.537 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707119156136019, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707119156, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "63d6440e-357a-4647-9b53-41365371aff9", "db_session_id": "03BOYWKNQXY5T4JHAMG3", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-05T07:45:56.537 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707119156136100, "job": 1, "event": "recovery_finished"} 2024-02-05T07:45:56.537 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-05T07:45:56.537 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-05T07:45:56.537 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557f3cc10000 2024-02-05T07:45:56.537 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: DB pointer 0x557f3cbfa000 2024-02-05T07:45:56.537 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-05T07:45:56.537 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-05T07:45:56.537 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** DB Stats ** 2024-02-05T07:45:56.537 INFO:journalctl@ceph.mon.c.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-05T07:45:56.537 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-05T07:45:56.537 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-05T07:45:56.537 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** Compaction Stats [default] ** 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** Compaction Stats [default] ** 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-05T07:45:56.538 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout: Block cache BinnedLRUCache@0x557f3cb1f090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: using public_addrv [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: starting mon.c rank -1 at public addrs [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] at bind addrs [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:45:56.539 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.c@-1(???) e0 preinit fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:45:56.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.c@-1(synchronizing).mds e1 new map 2024-02-05T07:45:56.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-05T07:45:56.540 INFO:journalctl@ceph.mon.c.smithi131.stdout: e1 2024-02-05T07:45:56.540 INFO:journalctl@ceph.mon.c.smithi131.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-05T07:45:56.540 INFO:journalctl@ceph.mon.c.smithi131.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-05T07:45:56.540 INFO:journalctl@ceph.mon.c.smithi131.stdout: legacy client fscid: -1 2024-02-05T07:45:56.540 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-02-05T07:45:56.540 INFO:journalctl@ceph.mon.c.smithi131.stdout: No filesystems configured 2024-02-05T07:45:56.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-05T07:45:56.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-05T07:45:56.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-05T07:45:56.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-05T07:45:56.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-05T07:45:56.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-05T07:45:56.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2130606218' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:45:56.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: Updating smithi157:/etc/ceph/ceph.conf 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2130606218' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: Updating smithi157:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: Updating smithi157:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:172.21.15.131=a;smithi131:[v2:172.21.15.131:3301,v1:172.21.15.131:6790]=c;smithi157:172.21.15.157=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: Saving service mon spec with placement smithi131:172.21.15.131=a;smithi131:[v2:172.21.15.131:3301,v1:172.21.15.131:6790]=c;smithi157:172.21.15.157=b;count:3 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:45:56.542 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:45:56.543 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: Deploying daemon mon.b on smithi157 2024-02-05T07:45:56.543 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:56.543 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:56.543 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:45:56.543 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:45:56.543 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.a calling monitor election 2024-02-05T07:45:56.543 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:45:56.543 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:56.543 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:45:56.543 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.b calling monitor election 2024-02-05T07:45:56.543 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:45:56.543 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:56.543 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:45:56.543 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:45:56.543 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-05T07:45:56.543 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: monmap e2: 2 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T07:45:56.544 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: fsmap 2024-02-05T07:45:56.544 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T07:45:56.544 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mgrmap e13: y(active, since 41s) 2024-02-05T07:45:56.544 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: overall HEALTH_OK 2024-02-05T07:45:56.544 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:56.544 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:56.544 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:45:56.544 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:45:56.544 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:45:56.544 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: Deploying daemon mon.c on smithi131 2024-02-05T07:45:56.544 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:56.544 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:45:56.544 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.157:0/642683668' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T07:45:56.544 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:45:56.544 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 ceph-mon[41654]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-05T07:45:56.544 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 bash[41616]: 72bec76edf25c9a73252ee7790d77c8c2e8eafe79d34854d2b931bf13ec48a24 2024-02-05T07:45:56.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:45:56 smithi131 systemd[1]: Started Ceph mon.c for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T07:46:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:46:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: mon.a calling monitor election 2024-02-05T07:46:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: mon.b calling monitor election 2024-02-05T07:46:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:46:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: mon.c calling monitor election 2024-02-05T07:46:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:01.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:01.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-05T07:46:01.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: monmap e3: 3 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0],c=[v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T07:46:01.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: fsmap 2024-02-05T07:46:01.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T07:46:01.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: mgrmap e13: y(active, since 50s) 2024-02-05T07:46:01.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: overall HEALTH_OK 2024-02-05T07:46:01.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:01.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:01.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:01.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:01.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:46:01.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: mon.a calling monitor election 2024-02-05T07:46:01.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: mon.b calling monitor election 2024-02-05T07:46:01.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:46:01.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:01.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: mon.c calling monitor election 2024-02-05T07:46:01.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:01.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:01.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-05T07:46:01.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: monmap e3: 3 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0],c=[v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T07:46:01.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: fsmap 2024-02-05T07:46:01.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T07:46:01.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: mgrmap e13: y(active, since 50s) 2024-02-05T07:46:01.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: overall HEALTH_OK 2024-02-05T07:46:01.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:01.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:01.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:01.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.524+0000 7fab96b16700 1 -- 172.21.15.157:0/1172619701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab800035c0 msgr2=0x7fab80005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.524+0000 7fab96b16700 1 --2- 172.21.15.157:0/1172619701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab800035c0 0x7fab80005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fab90100650 tx=0x7fab8400fe60 comp rx=0 tx=0).stop 2024-02-05T07:46:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.524+0000 7fab96b16700 1 -- 172.21.15.157:0/1172619701 shutdown_connections 2024-02-05T07:46:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.524+0000 7fab96b16700 1 --2- 172.21.15.157:0/1172619701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab800035c0 0x7fab80005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.524+0000 7fab96b16700 1 -- 172.21.15.157:0/1172619701 >> 172.21.15.157:0/1172619701 conn(0x7fab900fc230 msgr2=0x7fab900fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:46:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.524+0000 7fab96b16700 1 -- 172.21.15.157:0/1172619701 shutdown_connections 2024-02-05T07:46:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.524+0000 7fab96b16700 1 -- 172.21.15.157:0/1172619701 wait complete. 2024-02-05T07:46:01.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.525+0000 7fab96b16700 1 Processor -- start 2024-02-05T07:46:01.527 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:46:01.527 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: mon.a calling monitor election 2024-02-05T07:46:01.527 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: mon.b calling monitor election 2024-02-05T07:46:01.527 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:46:01.527 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.527 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:01.527 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.527 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.528 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: mon.c calling monitor election 2024-02-05T07:46:01.528 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:01.528 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.528 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.528 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:01.528 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:01.528 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-05T07:46:01.528 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: monmap e3: 3 mons at {a=[v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0],b=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0],c=[v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T07:46:01.528 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: fsmap 2024-02-05T07:46:01.528 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T07:46:01.528 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: mgrmap e13: y(active, since 50s) 2024-02-05T07:46:01.528 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: overall HEALTH_OK 2024-02-05T07:46:01.528 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:01.528 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:01.529 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:01.529 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:01.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.525+0000 7fab96b16700 1 -- start start 2024-02-05T07:46:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.525+0000 7fab96b16700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fab800035c0 0x7fab901022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.525+0000 7fab96b16700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab90102830 0x7fab90102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.525+0000 7fab96b16700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab901a2bf0 0x7fab901a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.525+0000 7fab96b16700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab901a55b0 con 0x7fab901a2bf0 2024-02-05T07:46:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.525+0000 7fab96b16700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab901a56f0 con 0x7fab90102830 2024-02-05T07:46:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.525+0000 7fab96b16700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab901a58b0 con 0x7fab800035c0 2024-02-05T07:46:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.525+0000 7fab8ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab90102830 0x7fab90102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.525+0000 7fab948b2700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fab800035c0 0x7fab901022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.525+0000 7fab948b2700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fab800035c0 0x7fab901022f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.157:55136/0 (socket says 172.21.15.157:55136) 2024-02-05T07:46:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.525+0000 7fab8ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab90102830 0x7fab90102cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54826/0 (socket says 172.21.15.157:54826) 2024-02-05T07:46:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab948b2700 1 -- 172.21.15.157:0/3023386433 learned_addr learned my addr 172.21.15.157:0/3023386433 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:46:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.525+0000 7fab950b3700 1 --2- 172.21.15.157:0/3023386433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab901a2bf0 0x7fab901a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab948b2700 1 -- 172.21.15.157:0/3023386433 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fab800035c0 msgr2=0x7fab901022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-05T07:46:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab948b2700 1 -- 172.21.15.157:0/3023386433 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fab800035c0 msgr2=0x7fab901022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-05T07:46:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab948b2700 1 --2- 172.21.15.157:0/3023386433 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fab800035c0 0x7fab901022f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-05T07:46:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab948b2700 1 --2- 172.21.15.157:0/3023386433 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fab800035c0 0x7fab901022f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-05T07:46:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab8ffff700 1 -- 172.21.15.157:0/3023386433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab90102830 msgr2=0x7fab90102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-05T07:46:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab8ffff700 1 -- 172.21.15.157:0/3023386433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab90102830 msgr2=0x7fab90102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-05T07:46:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab8ffff700 1 --2- 172.21.15.157:0/3023386433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab90102830 0x7fab90102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-05T07:46:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab8ffff700 1 --2- 172.21.15.157:0/3023386433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab90102830 0x7fab90102cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-05T07:46:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab950b3700 1 -- 172.21.15.157:0/3023386433 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fab800035c0 msgr2=0x7fab901022f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-05T07:46:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab950b3700 1 --2- 172.21.15.157:0/3023386433 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fab800035c0 0x7fab901022f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab950b3700 1 -- 172.21.15.157:0/3023386433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab90102830 msgr2=0x7fab90102cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-05T07:46:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab950b3700 1 --2- 172.21.15.157:0/3023386433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab90102830 0x7fab90102cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab950b3700 1 -- 172.21.15.157:0/3023386433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8400f8b0 con 0x7fab901a2bf0 2024-02-05T07:46:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab950b3700 1 --2- 172.21.15.157:0/3023386433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab901a2bf0 0x7fab901a4fe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab8800bd50 tx=0x7fab8800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:46:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab8dffb700 1 -- 172.21.15.157:0/3023386433 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8800cd10 con 0x7fab901a2bf0 2024-02-05T07:46:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab8dffb700 1 -- 172.21.15.157:0/3023386433 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fab8800ce70 con 0x7fab901a2bf0 2024-02-05T07:46:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab96b16700 1 -- 172.21.15.157:0/3023386433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab901a5b90 con 0x7fab901a2bf0 2024-02-05T07:46:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.526+0000 7fab8dffb700 1 -- 172.21.15.157:0/3023386433 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab880127e0 con 0x7fab901a2bf0 2024-02-05T07:46:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.527+0000 7fab96b16700 1 -- 172.21.15.157:0/3023386433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab901a6090 con 0x7fab901a2bf0 2024-02-05T07:46:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.527+0000 7fab8dffb700 1 -- 172.21.15.157:0/3023386433 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7fab88012960 con 0x7fab901a2bf0 2024-02-05T07:46:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.527+0000 7fab96b16700 1 -- 172.21.15.157:0/3023386433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab90190c70 con 0x7fab901a2bf0 2024-02-05T07:46:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.528+0000 7fab8dffb700 1 --2- 172.21.15.157:0/3023386433 >> v2:172.21.15.131:6800/760045032 conn(0x7fab740393d0 0x7fab7403b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.528+0000 7fab8dffb700 1 -- 172.21.15.157:0/3023386433 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fab8804c470 con 0x7fab901a2bf0 2024-02-05T07:46:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.528+0000 7fab948b2700 1 --2- 172.21.15.157:0/3023386433 >> v2:172.21.15.131:6800/760045032 conn(0x7fab740393d0 0x7fab7403b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:01.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.528+0000 7fab948b2700 1 --2- 172.21.15.157:0/3023386433 >> v2:172.21.15.131:6800/760045032 conn(0x7fab740393d0 0x7fab7403b890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fab84001eb0 tx=0x7fab84005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:46:01.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.537+0000 7fab8dffb700 1 -- 172.21.15.157:0/3023386433 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fab8801d3b0 con 0x7fab901a2bf0 2024-02-05T07:46:01.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.882+0000 7fab96b16700 1 -- 172.21.15.157:0/3023386433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fab90103870 con 0x7fab901a2bf0 2024-02-05T07:46:01.885 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.883+0000 7fab8dffb700 1 -- 172.21.15.157:0/3023386433 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1289 (secure 0 0 0) 0x7fab880117b0 con 0x7fab901a2bf0 2024-02-05T07:46:01.885 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-02-05T07:46:01.885 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":3,"fsid":"4018fbe8-c3fa-11ee-95b6-87774f69a715","modified":"2024-02-05T07:45:56.157728Z","created":"2024-02-05T07:44:04.773825Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3301","nonce":0},{"type":"v1","addr":"172.21.15.131:6790","nonce":0}]},"addr":"172.21.15.131:6790/0","public_addr":"172.21.15.131:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-02-05T07:46:01.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.886+0000 7fab96b16700 1 -- 172.21.15.157:0/3023386433 >> v2:172.21.15.131:6800/760045032 conn(0x7fab740393d0 msgr2=0x7fab7403b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:01.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.886+0000 7fab96b16700 1 --2- 172.21.15.157:0/3023386433 >> v2:172.21.15.131:6800/760045032 conn(0x7fab740393d0 0x7fab7403b890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fab84001eb0 tx=0x7fab84005a90 comp rx=0 tx=0).stop 2024-02-05T07:46:01.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.886+0000 7fab96b16700 1 -- 172.21.15.157:0/3023386433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab901a2bf0 msgr2=0x7fab901a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:01.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.886+0000 7fab96b16700 1 --2- 172.21.15.157:0/3023386433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab901a2bf0 0x7fab901a4fe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab8800bd50 tx=0x7fab8800be30 comp rx=0 tx=0).stop 2024-02-05T07:46:01.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.886+0000 7fab96b16700 1 -- 172.21.15.157:0/3023386433 shutdown_connections 2024-02-05T07:46:01.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.886+0000 7fab96b16700 1 --2- 172.21.15.157:0/3023386433 >> v2:172.21.15.131:6800/760045032 conn(0x7fab740393d0 0x7fab7403b890 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:01.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.886+0000 7fab96b16700 1 --2- 172.21.15.157:0/3023386433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab901a2bf0 0x7fab901a4fe0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:01.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.886+0000 7fab96b16700 1 --2- 172.21.15.157:0/3023386433 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fab800035c0 0x7fab901022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:01.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.886+0000 7fab96b16700 1 --2- 172.21.15.157:0/3023386433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab90102830 0x7fab90102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:01.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.886+0000 7fab96b16700 1 -- 172.21.15.157:0/3023386433 >> 172.21.15.157:0/3023386433 conn(0x7fab900fc230 msgr2=0x7fab9018ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:46:01.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.886+0000 7fab96b16700 1 -- 172.21.15.157:0/3023386433 shutdown_connections 2024-02-05T07:46:01.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:01.886+0000 7fab96b16700 1 -- 172.21.15.157:0/3023386433 wait complete. 2024-02-05T07:46:01.889 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 3 2024-02-05T07:46:02.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:02 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:02.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.157:0/3023386433' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T07:46:02.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:02 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:02.499 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-05T07:46:02.499 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph config generate-minimal-conf 2024-02-05T07:46:02.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:02 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:02.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.157:0/3023386433' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T07:46:02.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:02 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:02.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:02 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:02.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.157:0/3023386433' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T07:46:02.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:02 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:46:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:46:03.928 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[41654]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:03.928 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.928 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.929 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.929 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.929 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:03.929 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:03.929 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.929 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.929 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.929 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.929 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.929 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[29507]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:03.929 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.930 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.930 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.930 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.930 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:03.930 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:03.930 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.930 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.930 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.930 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.930 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:03 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:03 smithi157 ceph-mon[33345]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:03.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:03 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:03 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:03 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:03 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:03 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:03.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:03 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:03.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:03 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:03 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:03 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:03 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:03.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:03 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.450+0000 7f860fda6700 1 -- 172.21.15.131:0/252340304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8608106690 msgr2=0x7f8608106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.450+0000 7f860fda6700 1 --2- 172.21.15.131:0/252340304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8608106690 0x7f8608106a70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009910 tx=0x7f85f8009c20 comp rx=0 tx=0).stop 2024-02-05T07:46:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.450+0000 7f860fda6700 1 -- 172.21.15.131:0/252340304 shutdown_connections 2024-02-05T07:46:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.450+0000 7f860fda6700 1 --2- 172.21.15.131:0/252340304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8608106690 0x7f8608106a70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.450+0000 7f860fda6700 1 -- 172.21.15.131:0/252340304 >> 172.21.15.131:0/252340304 conn(0x7f86080fc130 msgr2=0x7f86080fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:46:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.450+0000 7f860fda6700 1 -- 172.21.15.131:0/252340304 shutdown_connections 2024-02-05T07:46:04.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.450+0000 7f860fda6700 1 -- 172.21.15.131:0/252340304 wait complete. 2024-02-05T07:46:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.451+0000 7f860fda6700 1 Processor -- start 2024-02-05T07:46:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.451+0000 7f860fda6700 1 -- start start 2024-02-05T07:46:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.451+0000 7f860fda6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8608106690 0x7f8608197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.451+0000 7f860fda6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8608197d50 0x7f86081981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.451+0000 7f860fda6700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f86081a0880 0x7f86081a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.451+0000 7f860fda6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081a3380 con 0x7f8608106690 2024-02-05T07:46:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.451+0000 7f860fda6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081a34f0 con 0x7f8608197d50 2024-02-05T07:46:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.451+0000 7f860fda6700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86081a36b0 con 0x7f86081a0880 2024-02-05T07:46:04.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.451+0000 7f860db42700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8608106690 0x7f8608197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:04.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.451+0000 7f860e343700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f86081a0880 0x7f86081a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:04.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.451+0000 7f860db42700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8608106690 0x7f8608197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57212/0 (socket says 172.21.15.131:57212) 2024-02-05T07:46:04.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.451+0000 7f860db42700 1 -- 172.21.15.131:0/2117399052 learned_addr learned my addr 172.21.15.131:0/2117399052 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:46:04.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.452+0000 7f860d341700 1 --2- 172.21.15.131:0/2117399052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8608197d50 0x7f86081981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:04.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.452+0000 7f860db42700 1 -- 172.21.15.131:0/2117399052 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f86081a0880 msgr2=0x7f86081a2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:04.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.452+0000 7f860db42700 1 --2- 172.21.15.131:0/2117399052 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f86081a0880 0x7f86081a2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:04.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.452+0000 7f860db42700 1 -- 172.21.15.131:0/2117399052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8608197d50 msgr2=0x7f86081981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:04.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.452+0000 7f860db42700 1 --2- 172.21.15.131:0/2117399052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8608197d50 0x7f86081981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:04.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.452+0000 7f860db42700 1 -- 172.21.15.131:0/2117399052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f80095f0 con 0x7f8608106690 2024-02-05T07:46:04.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.452+0000 7f860e343700 1 --2- 172.21.15.131:0/2117399052 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f86081a0880 0x7f86081a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:46:04.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.452+0000 7f860d341700 1 --2- 172.21.15.131:0/2117399052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8608197d50 0x7f86081981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-05T07:46:04.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.452+0000 7f860db42700 1 --2- 172.21.15.131:0/2117399052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8608106690 0x7f8608197810 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f85f8005950 tx=0x7f85f800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:46:04.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.452+0000 7f85feffd700 1 -- 172.21.15.131:0/2117399052 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f801d070 con 0x7f8608106690 2024-02-05T07:46:04.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.452+0000 7f85feffd700 1 -- 172.21.15.131:0/2117399052 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f85f80050d0 con 0x7f8608106690 2024-02-05T07:46:04.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.452+0000 7f860fda6700 1 -- 172.21.15.131:0/2117399052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86081a3990 con 0x7f8608106690 2024-02-05T07:46:04.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.452+0000 7f85feffd700 1 -- 172.21.15.131:0/2117399052 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f800fc40 con 0x7f8608106690 2024-02-05T07:46:04.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.452+0000 7f860fda6700 1 -- 172.21.15.131:0/2117399052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86081a3ee0 con 0x7f8608106690 2024-02-05T07:46:04.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.453+0000 7f85feffd700 1 -- 172.21.15.131:0/2117399052 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7f85f802b440 con 0x7f8608106690 2024-02-05T07:46:04.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.453+0000 7f860fda6700 1 -- 172.21.15.131:0/2117399052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8608105980 con 0x7f8608106690 2024-02-05T07:46:04.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.453+0000 7f85feffd700 1 --2- 172.21.15.131:0/2117399052 >> v2:172.21.15.131:6800/760045032 conn(0x7f85f40393d0 0x7f85f403b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:04.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.453+0000 7f85feffd700 1 -- 172.21.15.131:0/2117399052 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f85f804ce50 con 0x7f8608106690 2024-02-05T07:46:04.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.453+0000 7f860d341700 1 --2- 172.21.15.131:0/2117399052 >> v2:172.21.15.131:6800/760045032 conn(0x7f85f40393d0 0x7f85f403b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:04.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.454+0000 7f860d341700 1 --2- 172.21.15.131:0/2117399052 >> v2:172.21.15.131:6800/760045032 conn(0x7f85f40393d0 0x7f85f403b890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f86040079b0 tx=0x7f8604008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:46:04.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.458+0000 7f85feffd700 1 -- 172.21.15.131:0/2117399052 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f85f8050030 con 0x7f8608106690 2024-02-05T07:46:04.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.658+0000 7f860fda6700 1 -- 172.21.15.131:0/2117399052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f860806a120 con 0x7f8608106690 2024-02-05T07:46:04.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.659+0000 7f85feffd700 1 -- 172.21.15.131:0/2117399052 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f85f8020550 con 0x7f8608106690 2024-02-05T07:46:04.660 INFO:teuthology.orchestra.run.smithi131.stdout:# minimal ceph.conf for 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:46:04.660 INFO:teuthology.orchestra.run.smithi131.stdout:[global] 2024-02-05T07:46:04.660 INFO:teuthology.orchestra.run.smithi131.stdout: fsid = 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:46:04.660 INFO:teuthology.orchestra.run.smithi131.stdout: mon_host = [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] 2024-02-05T07:46:04.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.661+0000 7f860fda6700 1 -- 172.21.15.131:0/2117399052 >> v2:172.21.15.131:6800/760045032 conn(0x7f85f40393d0 msgr2=0x7f85f403b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:04.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.661+0000 7f860fda6700 1 --2- 172.21.15.131:0/2117399052 >> v2:172.21.15.131:6800/760045032 conn(0x7f85f40393d0 0x7f85f403b890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f86040079b0 tx=0x7f8604008040 comp rx=0 tx=0).stop 2024-02-05T07:46:04.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.661+0000 7f860fda6700 1 -- 172.21.15.131:0/2117399052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8608106690 msgr2=0x7f8608197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:04.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.661+0000 7f860fda6700 1 --2- 172.21.15.131:0/2117399052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8608106690 0x7f8608197810 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f85f8005950 tx=0x7f85f800bcf0 comp rx=0 tx=0).stop 2024-02-05T07:46:04.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.661+0000 7f860fda6700 1 -- 172.21.15.131:0/2117399052 shutdown_connections 2024-02-05T07:46:04.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.661+0000 7f860fda6700 1 --2- 172.21.15.131:0/2117399052 >> v2:172.21.15.131:6800/760045032 conn(0x7f85f40393d0 0x7f85f403b890 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:04.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.661+0000 7f860fda6700 1 --2- 172.21.15.131:0/2117399052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8608106690 0x7f8608197810 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:04.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.661+0000 7f860fda6700 1 --2- 172.21.15.131:0/2117399052 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f86081a0880 0x7f86081a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:04.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.661+0000 7f860fda6700 1 --2- 172.21.15.131:0/2117399052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8608197d50 0x7f86081981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:04.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.661+0000 7f860fda6700 1 -- 172.21.15.131:0/2117399052 >> 172.21.15.131:0/2117399052 conn(0x7f86080fc130 msgr2=0x7f86080fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:46:04.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.661+0000 7f860fda6700 1 -- 172.21.15.131:0/2117399052 shutdown_connections 2024-02-05T07:46:04.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:04.661+0000 7f860fda6700 1 -- 172.21.15.131:0/2117399052 wait complete. 2024-02-05T07:46:04.946 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[41654]: Updating smithi131:/etc/ceph/ceph.conf 2024-02-05T07:46:04.946 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[41654]: Updating smithi157:/etc/ceph/ceph.conf 2024-02-05T07:46:04.947 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[41654]: Updating smithi131:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:46:04.947 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[41654]: Updating smithi157:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:46:04.947 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:04.947 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:04.947 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:04.947 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:04.947 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[41654]: Reconfiguring mon.a (unknown last config time)... 2024-02-05T07:46:04.947 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:46:04.947 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:46:04.947 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:04.947 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[41654]: Reconfiguring daemon mon.a on smithi131 2024-02-05T07:46:04.947 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2117399052' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:04.947 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[29507]: Updating smithi131:/etc/ceph/ceph.conf 2024-02-05T07:46:04.948 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[29507]: Updating smithi157:/etc/ceph/ceph.conf 2024-02-05T07:46:04.948 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[29507]: Updating smithi131:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:46:04.948 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[29507]: Updating smithi157:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:46:04.948 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:04.948 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:04.948 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:04.948 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:04.948 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[29507]: Reconfiguring mon.a (unknown last config time)... 2024-02-05T07:46:04.948 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:46:04.948 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:46:04.948 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:04.948 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[29507]: Reconfiguring daemon mon.a on smithi131 2024-02-05T07:46:04.948 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2117399052' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:04.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:04 smithi157 ceph-mon[33345]: Updating smithi131:/etc/ceph/ceph.conf 2024-02-05T07:46:04.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:04 smithi157 ceph-mon[33345]: Updating smithi157:/etc/ceph/ceph.conf 2024-02-05T07:46:04.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:04 smithi157 ceph-mon[33345]: Updating smithi131:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:46:04.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:04 smithi157 ceph-mon[33345]: Updating smithi157:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:46:04.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:04 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:04.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:04 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:04.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:04 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:04.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:04 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:04.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:04 smithi157 ceph-mon[33345]: Reconfiguring mon.a (unknown last config time)... 2024-02-05T07:46:04.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:04 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:46:04.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:04 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:46:04.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:04 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:04.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:04 smithi157 ceph-mon[33345]: Reconfiguring daemon mon.a on smithi131 2024-02-05T07:46:04.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2117399052' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:05.240 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-05T07:46:05.240 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:46:05.241 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.conf 2024-02-05T07:46:05.310 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:46:05.310 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:46:05.378 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-02-05T07:46:05.378 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.conf 2024-02-05T07:46:05.413 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-02-05T07:46:05.413 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:46:05.484 INFO:tasks.cephadm:Adding mgr.y on smithi131 2024-02-05T07:46:05.484 INFO:tasks.cephadm:Adding mgr.x on smithi157 2024-02-05T07:46:05.484 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch apply mgr '2;smithi131=y;smithi157=x' 2024-02-05T07:46:05.756 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:46:05.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:05 smithi131 ceph-mon[29507]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:05.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:05.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:05 smithi131 ceph-mon[41654]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:05.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:06.005 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:05 smithi157 ceph-mon[33345]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:06.005 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:07.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.182+0000 7f9787250700 1 -- 172.21.15.157:0/1190344591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9780102f70 msgr2=0x7f9780103390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:07.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.182+0000 7f9787250700 1 --2- 172.21.15.157:0/1190344591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9780102f70 0x7f9780103390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f977800b0a0 tx=0x7f977800b3b0 comp rx=0 tx=0).stop 2024-02-05T07:46:07.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.182+0000 7f9787250700 1 -- 172.21.15.157:0/1190344591 shutdown_connections 2024-02-05T07:46:07.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.182+0000 7f9787250700 1 --2- 172.21.15.157:0/1190344591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9780102f70 0x7f9780103390 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:07.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.182+0000 7f9787250700 1 -- 172.21.15.157:0/1190344591 >> 172.21.15.157:0/1190344591 conn(0x7f97800fce10 msgr2=0x7f97800ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:46:07.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.182+0000 7f9787250700 1 -- 172.21.15.157:0/1190344591 shutdown_connections 2024-02-05T07:46:07.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.182+0000 7f9787250700 1 -- 172.21.15.157:0/1190344591 wait complete. 2024-02-05T07:46:07.186 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.182+0000 7f9787250700 1 Processor -- start 2024-02-05T07:46:07.186 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.182+0000 7f9787250700 1 -- start start 2024-02-05T07:46:07.186 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.182+0000 7f9787250700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9780102f70 0x7f9780196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f9787250700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9780196b40 0x7f978019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f9787250700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9780196fc0 0x7f978019e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f9787250700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978019e8e0 con 0x7f9780102f70 2024-02-05T07:46:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f9787250700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978019ea50 con 0x7f9780196fc0 2024-02-05T07:46:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f9787250700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978019ec10 con 0x7f9780196b40 2024-02-05T07:46:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f9784fec700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9780102f70 0x7f9780196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f97857ed700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9780196fc0 0x7f978019e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f9784fec700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9780102f70 0x7f9780196600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.157:56108/0 (socket says 172.21.15.157:56108) 2024-02-05T07:46:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f9784fec700 1 -- 172.21.15.157:0/1491340229 learned_addr learned my addr 172.21.15.157:0/1491340229 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:46:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f977ffff700 1 --2- 172.21.15.157:0/1491340229 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9780196b40 0x7f978019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f9784fec700 1 -- 172.21.15.157:0/1491340229 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9780196b40 msgr2=0x7f978019bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f9784fec700 1 --2- 172.21.15.157:0/1491340229 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9780196b40 0x7f978019bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f9784fec700 1 -- 172.21.15.157:0/1491340229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9780196fc0 msgr2=0x7f978019e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f9784fec700 1 --2- 172.21.15.157:0/1491340229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9780196fc0 0x7f978019e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f9784fec700 1 -- 172.21.15.157:0/1491340229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9778009d00 con 0x7f9780102f70 2024-02-05T07:46:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f9784fec700 1 --2- 172.21.15.157:0/1491340229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9780102f70 0x7f9780196600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f977800d010 tx=0x7f977800bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:46:07.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f977dffb700 1 -- 172.21.15.157:0/1491340229 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977800e040 con 0x7f9780102f70 2024-02-05T07:46:07.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f977dffb700 1 -- 172.21.15.157:0/1491340229 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f97780079d0 con 0x7f9780102f70 2024-02-05T07:46:07.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f977dffb700 1 -- 172.21.15.157:0/1491340229 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9778012b00 con 0x7f9780102f70 2024-02-05T07:46:07.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.183+0000 7f9787250700 1 -- 172.21.15.157:0/1491340229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f978019ee90 con 0x7f9780102f70 2024-02-05T07:46:07.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.184+0000 7f9787250700 1 -- 172.21.15.157:0/1491340229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978019f330 con 0x7f9780102f70 2024-02-05T07:46:07.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.184+0000 7f977dffb700 1 -- 172.21.15.157:0/1491340229 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7f9778019070 con 0x7f9780102f70 2024-02-05T07:46:07.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.184+0000 7f9787250700 1 -- 172.21.15.157:0/1491340229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97801904a0 con 0x7f9780102f70 2024-02-05T07:46:07.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.185+0000 7f977dffb700 1 --2- 172.21.15.157:0/1491340229 >> v2:172.21.15.131:6800/760045032 conn(0x7f976c0393d0 0x7f976c03b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:07.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.185+0000 7f977dffb700 1 -- 172.21.15.157:0/1491340229 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f977804cf10 con 0x7f9780102f70 2024-02-05T07:46:07.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.185+0000 7f977ffff700 1 --2- 172.21.15.157:0/1491340229 >> v2:172.21.15.131:6800/760045032 conn(0x7f976c0393d0 0x7f976c03b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:07.188 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.185+0000 7f977ffff700 1 --2- 172.21.15.157:0/1491340229 >> v2:172.21.15.131:6800/760045032 conn(0x7f976c0393d0 0x7f976c03b890 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9770009ad0 tx=0x7f9770009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:46:07.191 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.189+0000 7f977dffb700 1 -- 172.21.15.157:0/1491340229 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f97780059c0 con 0x7f9780102f70 2024-02-05T07:46:07.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:07 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:07.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:07 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:07.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:07 smithi131 ceph-mon[29507]: Reconfiguring mon.c (monmap changed)... 2024-02-05T07:46:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:07 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:46:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:07 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:46:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:07 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:07 smithi131 ceph-mon[29507]: Reconfiguring daemon mon.c on smithi131 2024-02-05T07:46:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:07 smithi131 ceph-mon[29507]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:07 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:07 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:07 smithi131 ceph-mon[41654]: Reconfiguring mon.c (monmap changed)... 2024-02-05T07:46:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:07 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:46:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:07 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:46:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:07 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:07 smithi131 ceph-mon[41654]: Reconfiguring daemon mon.c on smithi131 2024-02-05T07:46:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:07 smithi131 ceph-mon[41654]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:07.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.395+0000 7f9787250700 1 -- 172.21.15.157:0/1491340229 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi131=y;smithi157=x", "target": ["mon-mgr", ""]}) v1 -- 0x7f9780062160 con 0x7f976c0393d0 2024-02-05T07:46:07.397 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:07 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:07.397 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:07 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:07.397 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:07 smithi157 ceph-mon[33345]: Reconfiguring mon.c (monmap changed)... 2024-02-05T07:46:07.397 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:07 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:46:07.397 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:07 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:46:07.397 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:07 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:07.397 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:07 smithi157 ceph-mon[33345]: Reconfiguring daemon mon.c on smithi131 2024-02-05T07:46:07.398 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:07 smithi157 ceph-mon[33345]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:07.402 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.401+0000 7f977dffb700 1 -- 172.21.15.157:0/1491340229 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9780062160 con 0x7f976c0393d0 2024-02-05T07:46:07.403 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mgr update... 2024-02-05T07:46:07.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.403+0000 7f9787250700 1 -- 172.21.15.157:0/1491340229 >> v2:172.21.15.131:6800/760045032 conn(0x7f976c0393d0 msgr2=0x7f976c03b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:07.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.403+0000 7f9787250700 1 --2- 172.21.15.157:0/1491340229 >> v2:172.21.15.131:6800/760045032 conn(0x7f976c0393d0 0x7f976c03b890 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9770009ad0 tx=0x7f9770009210 comp rx=0 tx=0).stop 2024-02-05T07:46:07.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.403+0000 7f9787250700 1 -- 172.21.15.157:0/1491340229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9780102f70 msgr2=0x7f9780196600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:07.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.403+0000 7f9787250700 1 --2- 172.21.15.157:0/1491340229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9780102f70 0x7f9780196600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f977800d010 tx=0x7f977800bd10 comp rx=0 tx=0).stop 2024-02-05T07:46:07.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.403+0000 7f9787250700 1 -- 172.21.15.157:0/1491340229 shutdown_connections 2024-02-05T07:46:07.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.403+0000 7f9787250700 1 --2- 172.21.15.157:0/1491340229 >> v2:172.21.15.131:6800/760045032 conn(0x7f976c0393d0 0x7f976c03b890 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:07.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.403+0000 7f9787250700 1 --2- 172.21.15.157:0/1491340229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9780102f70 0x7f9780196600 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:07.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.403+0000 7f9787250700 1 --2- 172.21.15.157:0/1491340229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9780196fc0 0x7f978019e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:07.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.403+0000 7f9787250700 1 --2- 172.21.15.157:0/1491340229 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9780196b40 0x7f978019bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:07.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.403+0000 7f9787250700 1 -- 172.21.15.157:0/1491340229 >> 172.21.15.157:0/1491340229 conn(0x7f97800fce10 msgr2=0x7f97800fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:46:07.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.403+0000 7f9787250700 1 -- 172.21.15.157:0/1491340229 shutdown_connections 2024-02-05T07:46:07.406 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:46:07.403+0000 7f9787250700 1 -- 172.21.15.157:0/1491340229 wait complete. 2024-02-05T07:46:08.022 DEBUG:teuthology.orchestra.run.smithi157:mgr.x> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mgr.x.service 2024-02-05T07:46:08.024 INFO:tasks.cephadm:Deploying OSDs... 2024-02-05T07:46:08.024 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:46:08.024 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2024-02-05T07:46:08.040 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-05T07:46:08.040 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2024-02-05T07:46:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-05T07:46:08.094 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T07:46:08.094 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-05T07:46:08.094 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T07:46:08.094 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T07:46:08.094 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-02-05 07:45:55.506403437 +0000 2024-02-05T07:46:08.095 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-02-05 07:40:17.908126682 +0000 2024-02-05T07:46:08.095 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-02-05 07:40:17.908126682 +0000 2024-02-05T07:46:08.095 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-02-05 07:40:17.908126682 +0000 2024-02-05T07:46:08.095 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-05T07:46:08.161 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-02-05T07:46:08.161 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-02-05T07:46:08.161 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000267557 s, 1.9 MB/s 2024-02-05T07:46:08.162 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-05T07:46:08.218 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2024-02-05T07:46:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-05T07:46:08.273 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T07:46:08.273 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-05T07:46:08.274 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T07:46:08.274 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T07:46:08.274 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-02-05 07:45:55.506403437 +0000 2024-02-05T07:46:08.274 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-02-05 07:40:18.255119073 +0000 2024-02-05T07:46:08.274 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-02-05 07:40:18.255119073 +0000 2024-02-05T07:46:08.274 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-02-05 07:40:18.255119073 +0000 2024-02-05T07:46:08.274 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-05T07:46:08.339 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-02-05T07:46:08.339 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-02-05T07:46:08.339 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000191158 s, 2.7 MB/s 2024-02-05T07:46:08.341 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-05T07:46:08.396 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2024-02-05T07:46:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-05T07:46:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T07:46:08.451 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-05T07:46:08.451 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T07:46:08.451 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T07:46:08.451 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-02-05 07:45:55.507403416 +0000 2024-02-05T07:46:08.451 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-02-05 07:40:18.619111091 +0000 2024-02-05T07:46:08.451 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-02-05 07:40:18.619111091 +0000 2024-02-05T07:46:08.451 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-02-05 07:40:18.619111091 +0000 2024-02-05T07:46:08.451 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-05T07:46:08.514 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:08 smithi131 ceph-mon[29507]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi131=y;smithi157=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:46:08.514 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:08 smithi131 ceph-mon[29507]: Saving service mgr spec with placement smithi131=y;smithi157=x;count:2 2024-02-05T07:46:08.514 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:08 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:08.514 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:08 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:08.514 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:08 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:08.514 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:08 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:46:08.514 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:08 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:46:08.514 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:08 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:08.514 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:08 smithi131 ceph-mon[41654]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi131=y;smithi157=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:46:08.514 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:08 smithi131 ceph-mon[41654]: Saving service mgr spec with placement smithi131=y;smithi157=x;count:2 2024-02-05T07:46:08.515 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:08 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:08.515 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:08 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:08.515 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:08 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:08.515 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:08 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:46:08.515 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:08 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:46:08.515 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:08 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:08.516 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-02-05T07:46:08.516 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-02-05T07:46:08.516 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000168826 s, 3.0 MB/s 2024-02-05T07:46:08.517 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-05T07:46:08.573 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2024-02-05T07:46:08.628 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-05T07:46:08.628 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T07:46:08.628 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-05T07:46:08.628 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T07:46:08.628 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T07:46:08.628 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-02-05 07:45:55.507403416 +0000 2024-02-05T07:46:08.629 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-02-05 07:40:18.955103723 +0000 2024-02-05T07:46:08.629 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-02-05 07:40:18.955103723 +0000 2024-02-05T07:46:08.629 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-02-05 07:40:18.955103723 +0000 2024-02-05T07:46:08.629 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-05T07:46:08.694 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-02-05T07:46:08.694 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-02-05T07:46:08.694 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000181774 s, 2.8 MB/s 2024-02-05T07:46:08.695 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-05T07:46:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:08 smithi157 ceph-mon[33345]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi131=y;smithi157=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:46:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:08 smithi157 ceph-mon[33345]: Saving service mgr spec with placement smithi131=y;smithi157=x;count:2 2024-02-05T07:46:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:08 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:08 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:08 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:08 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:46:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:08 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:46:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:08 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:08.751 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-02-05T07:46:08.751 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-02-05T07:46:08.767 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-05T07:46:08.767 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-02-05T07:46:08.822 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-05T07:46:08.822 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T07:46:08.822 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-05T07:46:08.822 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T07:46:08.822 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T07:46:08.822 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-02-05 07:45:46.443105146 +0000 2024-02-05T07:46:08.822 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-02-05 07:40:17.896070886 +0000 2024-02-05T07:46:08.822 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-02-05 07:40:17.896070886 +0000 2024-02-05T07:46:08.823 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-02-05 07:40:17.896070886 +0000 2024-02-05T07:46:08.823 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-05T07:46:08.889 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-02-05T07:46:08.889 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-02-05T07:46:08.889 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 8.7761e-05 s, 5.8 MB/s 2024-02-05T07:46:08.890 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-05T07:46:08.946 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-02-05T07:46:09.001 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-05T07:46:09.001 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T07:46:09.001 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-05T07:46:09.001 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T07:46:09.002 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T07:46:09.002 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-02-05 07:45:46.443105146 +0000 2024-02-05T07:46:09.002 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-02-05 07:40:18.233065419 +0000 2024-02-05T07:46:09.002 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-02-05 07:40:18.233065419 +0000 2024-02-05T07:46:09.002 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-02-05 07:40:18.233065419 +0000 2024-02-05T07:46:09.002 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-05T07:46:09.066 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-02-05T07:46:09.067 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-02-05T07:46:09.067 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000109139 s, 4.7 MB/s 2024-02-05T07:46:09.068 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-05T07:46:09.123 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-02-05T07:46:09.178 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-05T07:46:09.178 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T07:46:09.179 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-02-05T07:46:09.179 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T07:46:09.179 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T07:46:09.179 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-02-05 07:45:46.443105146 +0000 2024-02-05T07:46:09.179 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-02-05 07:40:18.572059918 +0000 2024-02-05T07:46:09.179 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-02-05 07:40:18.572059918 +0000 2024-02-05T07:46:09.179 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-02-05 07:40:18.572059918 +0000 2024-02-05T07:46:09.179 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-05T07:46:09.243 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-02-05T07:46:09.244 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-02-05T07:46:09.244 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000102761 s, 5.0 MB/s 2024-02-05T07:46:09.245 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-05T07:46:09.300 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-02-05T07:46:09.356 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-05T07:46:09.356 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T07:46:09.359 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-02-05T07:46:09.359 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T07:46:09.359 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T07:46:09.359 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-02-05 07:45:46.444105130 +0000 2024-02-05T07:46:09.359 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-02-05 07:40:18.907054483 +0000 2024-02-05T07:46:09.359 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-02-05 07:40:18.907054483 +0000 2024-02-05T07:46:09.359 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-02-05 07:40:18.907054483 +0000 2024-02-05T07:46:09.359 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-05T07:46:09.422 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-02-05T07:46:09.423 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-02-05T07:46:09.423 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 8.3153e-05 s, 6.2 MB/s 2024-02-05T07:46:09.424 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-05T07:46:09.480 INFO:tasks.cephadm:Deploying osd.0 on smithi131 with /dev/vg_nvme/lv_4... 2024-02-05T07:46:09.480 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-05T07:46:09.719 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:46:09.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:09 smithi157 ceph-mon[33345]: Reconfiguring mon.b (monmap changed)... 2024-02-05T07:46:09.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:09 smithi157 ceph-mon[33345]: Reconfiguring daemon mon.b on smithi157 2024-02-05T07:46:09.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:09 smithi157 ceph-mon[33345]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:09 smithi131 ceph-mon[29507]: Reconfiguring mon.b (monmap changed)... 2024-02-05T07:46:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:09 smithi131 ceph-mon[29507]: Reconfiguring daemon mon.b on smithi157 2024-02-05T07:46:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:09 smithi131 ceph-mon[29507]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:09.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:09 smithi131 ceph-mon[41654]: Reconfiguring mon.b (monmap changed)... 2024-02-05T07:46:09.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:09 smithi131 ceph-mon[41654]: Reconfiguring daemon mon.b on smithi157 2024-02-05T07:46:09.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:09 smithi131 ceph-mon[41654]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:10.708 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:10 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:10.708 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:10 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:10.708 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:10 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:10.708 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:10 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:10.708 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:10 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:10.708 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:10 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:10.708 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:10 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T07:46:10.709 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:10 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-05T07:46:10.709 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:10 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T07:46:10.709 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:10 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:10.709 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:10 smithi157 ceph-mon[33345]: Deploying daemon mgr.x on smithi157 2024-02-05T07:46:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T07:46:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-05T07:46:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T07:46:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[29507]: Deploying daemon mgr.x on smithi157 2024-02-05T07:46:10.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:10.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:10.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:10.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:10.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:10.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:10.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T07:46:10.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-05T07:46:10.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T07:46:10.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:10.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:10 smithi131 ceph-mon[41654]: Deploying daemon mgr.x on smithi157 2024-02-05T07:46:11.367 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:11 smithi157 systemd[1]: Starting Ceph mgr.x for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T07:46:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:11 smithi131 ceph-mon[41654]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:11.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:11 smithi131 ceph-mon[29507]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:11.699 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:11 smithi157 ceph-mon[33345]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:11.699 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:11 smithi157 podman[35418]: 2024-02-05 07:46:11.486299327 +0000 UTC m=+0.017608682 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T07:46:11.699 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:11 smithi157 podman[35418]: 2024-02-05 07:46:11.593661254 +0000 UTC m=+0.124970600 container create 36d1da2f2cb7f7de54a1da26484926e03002d1008ed0e1b5e056bd14bcbd9680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T07:46:11.760 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-05T07:46:11.760 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-05T07:46:11.760 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-02-05T07:46:11.760 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-02-05T07:46:11.760 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112664 s, 931 MB/s 2024-02-05T07:46:11.760 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-02-05T07:46:12.223 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:11 smithi157 podman[35418]: 2024-02-05 07:46:11.796214835 +0000 UTC m=+0.327524181 container init 36d1da2f2cb7f7de54a1da26484926e03002d1008ed0e1b5e056bd14bcbd9680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.build-date=20240131) 2024-02-05T07:46:12.223 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:11 smithi157 podman[35418]: 2024-02-05 07:46:11.801297221 +0000 UTC m=+0.332606567 container start 36d1da2f2cb7f7de54a1da26484926e03002d1008ed0e1b5e056bd14bcbd9680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-05T07:46:12.224 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:11 smithi157 ceph-mgr[35456]: -- 172.21.15.157:0/4053834068 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556af8349520 con 0x556af831e800 2024-02-05T07:46:12.224 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:11 smithi157 bash[35418]: 36d1da2f2cb7f7de54a1da26484926e03002d1008ed0e1b5e056bd14bcbd9680 2024-02-05T07:46:12.224 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:11 smithi157 systemd[1]: Started Ceph mgr.x for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T07:46:12.391 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi131:vg_nvme/lv_4 2024-02-05T07:46:12.626 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:46:12.721 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:12 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:12.367+0000 7f8d6c947200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T07:46:12.721 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:12 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:12.464+0000 7f8d6c947200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T07:46:13.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:12 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:13.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:12 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:13.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:12 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:13.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:12 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:13.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:12 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:13.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:12 smithi131 ceph-mon[29507]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:13.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:12 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:13.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:12 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:13.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:12 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:13.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:12 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:13.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:12 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:13.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:12 smithi131 ceph-mon[41654]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:13.221 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:13 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:13.148+0000 7f8d6c947200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T07:46:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:12 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:12 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:12 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:12 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:12 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:12 smithi157 ceph-mon[33345]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:13.578 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:13 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:13.242+0000 7f8d6c947200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T07:46:13.578 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:13 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:13.389+0000 7f8d6c947200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T07:46:13.578 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:13 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:13.490+0000 7f8d6c947200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T07:46:13.578 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:13 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:13.576+0000 7f8d6c947200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T07:46:14.130 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:13 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:13.840+0000 7f8d6c947200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T07:46:14.130 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:13 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:13.949+0000 7f8d6c947200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T07:46:14.130 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:14 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:14.034+0000 7f8d6c947200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T07:46:14.130 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:14 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:14.128+0000 7f8d6c947200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T07:46:14.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.310+0000 7f55f3b39700 1 -- 172.21.15.131:0/2099735855 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f55ec105110 msgr2=0x7f55ec107500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:14.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.311+0000 7f55f3b39700 1 --2- 172.21.15.131:0/2099735855 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f55ec105110 0x7f55ec107500 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0098f0 tx=0x7f55dc009c00 comp rx=0 tx=0).stop 2024-02-05T07:46:14.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.311+0000 7f55f3b39700 1 -- 172.21.15.131:0/2099735855 shutdown_connections 2024-02-05T07:46:14.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.311+0000 7f55f3b39700 1 --2- 172.21.15.131:0/2099735855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55ec107a40 0x7f55ec109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:14.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.311+0000 7f55f3b39700 1 --2- 172.21.15.131:0/2099735855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55ec1027e0 0x7f55ec104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:14.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.311+0000 7f55f3b39700 1 --2- 172.21.15.131:0/2099735855 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f55ec105110 0x7f55ec107500 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:14.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.311+0000 7f55f3b39700 1 -- 172.21.15.131:0/2099735855 >> 172.21.15.131:0/2099735855 conn(0x7f55ec0fc210 msgr2=0x7f55ec0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:46:14.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.311+0000 7f55f3b39700 1 -- 172.21.15.131:0/2099735855 shutdown_connections 2024-02-05T07:46:14.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.311+0000 7f55f3b39700 1 -- 172.21.15.131:0/2099735855 wait complete. 2024-02-05T07:46:14.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.311+0000 7f55f3b39700 1 Processor -- start 2024-02-05T07:46:14.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.311+0000 7f55f3b39700 1 -- start start 2024-02-05T07:46:14.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.311+0000 7f55f3b39700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55ec1027e0 0x7f55ec197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:14.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.311+0000 7f55f3b39700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55ec105110 0x7f55ec197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:14.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.311+0000 7f55f3b39700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f55ec107a40 0x7f55ec19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:14.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.311+0000 7f55f3b39700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec19efc0 con 0x7f55ec105110 2024-02-05T07:46:14.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.311+0000 7f55f3b39700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec19f100 con 0x7f55ec1027e0 2024-02-05T07:46:14.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.311+0000 7f55f3b39700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec19f290 con 0x7f55ec107a40 2024-02-05T07:46:14.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55f10d4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55ec105110 0x7f55ec197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:14.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55f20d6700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f55ec107a40 0x7f55ec19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:14.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55f18d5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55ec1027e0 0x7f55ec197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:14.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55f10d4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55ec105110 0x7f55ec197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60026/0 (socket says 172.21.15.131:60026) 2024-02-05T07:46:14.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55f20d6700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f55ec107a40 0x7f55ec19e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:53260/0 (socket says 172.21.15.131:53260) 2024-02-05T07:46:14.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55f10d4700 1 -- 172.21.15.131:0/2093630350 learned_addr learned my addr 172.21.15.131:0/2093630350 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:46:14.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55f10d4700 1 -- 172.21.15.131:0/2093630350 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f55ec107a40 msgr2=0x7f55ec19e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:14.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55f10d4700 1 --2- 172.21.15.131:0/2093630350 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f55ec107a40 0x7f55ec19e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:14.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55f10d4700 1 -- 172.21.15.131:0/2093630350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55ec1027e0 msgr2=0x7f55ec197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:14.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55f10d4700 1 --2- 172.21.15.131:0/2093630350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55ec1027e0 0x7f55ec197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:14.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55f10d4700 1 -- 172.21.15.131:0/2093630350 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e4009d00 con 0x7f55ec105110 2024-02-05T07:46:14.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55f10d4700 1 --2- 172.21.15.131:0/2093630350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55ec105110 0x7f55ec197970 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009230 tx=0x7f55dc00b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:46:14.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55e2ffd700 1 -- 172.21.15.131:0/2093630350 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc01c070 con 0x7f55ec105110 2024-02-05T07:46:14.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55f3b39700 1 -- 172.21.15.131:0/2093630350 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55dc009520 con 0x7f55ec105110 2024-02-05T07:46:14.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55e2ffd700 1 -- 172.21.15.131:0/2093630350 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f55dc0054f0 con 0x7f55ec105110 2024-02-05T07:46:14.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55e2ffd700 1 -- 172.21.15.131:0/2093630350 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc004c90 con 0x7f55ec105110 2024-02-05T07:46:14.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.312+0000 7f55f3b39700 1 -- 172.21.15.131:0/2093630350 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ec19f790 con 0x7f55ec105110 2024-02-05T07:46:14.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.313+0000 7f55e2ffd700 1 -- 172.21.15.131:0/2093630350 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7f55dc005040 con 0x7f55ec105110 2024-02-05T07:46:14.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.313+0000 7f55e2ffd700 1 --2- 172.21.15.131:0/2093630350 >> v2:172.21.15.131:6800/760045032 conn(0x7f55d8039180 0x7f55d803b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:14.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.313+0000 7f55e2ffd700 1 -- 172.21.15.131:0/2093630350 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f55dc04cfd0 con 0x7f55ec105110 2024-02-05T07:46:14.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.313+0000 7f55f3b39700 1 -- 172.21.15.131:0/2093630350 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55ec0fdd90 con 0x7f55ec105110 2024-02-05T07:46:14.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.313+0000 7f55f18d5700 1 --2- 172.21.15.131:0/2093630350 >> v2:172.21.15.131:6800/760045032 conn(0x7f55d8039180 0x7f55d803b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:14.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.314+0000 7f55f18d5700 1 --2- 172.21.15.131:0/2093630350 >> v2:172.21.15.131:6800/760045032 conn(0x7f55d8039180 0x7f55d803b640 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f55e8005950 tx=0x7f55e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:46:14.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.318+0000 7f55e2ffd700 1 -- 172.21.15.131:0/2093630350 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f55dc00ba20 con 0x7f55ec105110 2024-02-05T07:46:14.471 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:14 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:14.305+0000 7f8d6c947200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T07:46:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:14.527+0000 7f55f3b39700 1 -- 172.21.15.131:0/2093630350 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f55ec062160 con 0x7f55d8039180 2024-02-05T07:46:14.720 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:14.720 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:14.720 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:14.720 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:14.720 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:14.720 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[41654]: Reconfiguring mgr.y (unknown last config time)... 2024-02-05T07:46:14.720 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T07:46:14.720 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T07:46:14.721 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:14.721 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[41654]: Reconfiguring daemon mgr.y on smithi131 2024-02-05T07:46:14.721 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:14.721 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:14.721 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:14.721 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:14.721 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:14.721 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[29507]: Reconfiguring mgr.y (unknown last config time)... 2024-02-05T07:46:14.721 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T07:46:14.721 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T07:46:14.721 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:14.721 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:14 smithi131 ceph-mon[29507]: Reconfiguring daemon mgr.y on smithi131 2024-02-05T07:46:14.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:14 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:14.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:14 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:14.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:14 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:14.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:14 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:14.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:14 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:14.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:14 smithi157 ceph-mon[33345]: Reconfiguring mgr.y (unknown last config time)... 2024-02-05T07:46:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:14 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T07:46:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:14 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T07:46:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:14 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:14 smithi157 ceph-mon[33345]: Reconfiguring daemon mgr.y on smithi131 2024-02-05T07:46:15.721 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:15 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:15.269+0000 7f8d6c947200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T07:46:15.721 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:15 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:15.398+0000 7f8d6c947200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T07:46:15.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:15 smithi157 ceph-mon[33345]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:46:15.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:15 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:46:15.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:15 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:46:15.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:15 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:15.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:15 smithi157 ceph-mon[33345]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:15.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:15 smithi131 ceph-mon[29507]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:46:15.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:15 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:46:15.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:15 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:46:15.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:15 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:15.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:15 smithi131 ceph-mon[29507]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:15.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:15 smithi131 ceph-mon[41654]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:46:15.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:15 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:46:15.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:15 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:46:15.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:15 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:15.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:15 smithi131 ceph-mon[41654]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:16.016 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:15 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:15.732+0000 7f8d6c947200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T07:46:16.017 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:15 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:15.822+0000 7f8d6c947200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T07:46:16.017 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:16 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:16.014+0000 7f8d6c947200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:46:16.471 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:16 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:16.196+0000 7f8d6c947200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T07:46:16.744 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:16 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:16.744 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:16 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:16.744 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:16 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:16.744 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:16 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:16.744 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:16 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:16.476+0000 7f8d6c947200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:46:16.744 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:16 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:16.741+0000 7f8d6c947200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T07:46:16.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:16 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:16.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:16 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:16.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:16 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:16.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:16 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:16.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:16 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:16.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:16 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:16.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:16 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:16.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:16 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:17.221 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:16 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:16.976+0000 7f8d6c947200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T07:46:17.971 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:17 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:17.772+0000 7f8d6c947200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T07:46:17.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:17 smithi157 ceph-mon[33345]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:17.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:17 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:17.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:17 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:17.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:17 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:17.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3791383468' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f89dd964-ec08-4f4b-9e8f-91bea0156d35"}]: dispatch 2024-02-05T07:46:17.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3791383468' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f89dd964-ec08-4f4b-9e8f-91bea0156d35"}]': finished 2024-02-05T07:46:17.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:17 smithi157 ceph-mon[33345]: osdmap e5: 1 total, 0 up, 1 in 2024-02-05T07:46:17.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:17 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:46:18.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:17 smithi131 ceph-mon[29507]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:18.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:17 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:18.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:17 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:18.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:17 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:18.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3791383468' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f89dd964-ec08-4f4b-9e8f-91bea0156d35"}]: dispatch 2024-02-05T07:46:18.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3791383468' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f89dd964-ec08-4f4b-9e8f-91bea0156d35"}]': finished 2024-02-05T07:46:18.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:17 smithi131 ceph-mon[29507]: osdmap e5: 1 total, 0 up, 1 in 2024-02-05T07:46:18.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:17 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:46:18.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:17 smithi131 ceph-mon[41654]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:18.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:17 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:18.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:17 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:18.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:17 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:18.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3791383468' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f89dd964-ec08-4f4b-9e8f-91bea0156d35"}]: dispatch 2024-02-05T07:46:18.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3791383468' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f89dd964-ec08-4f4b-9e8f-91bea0156d35"}]': finished 2024-02-05T07:46:18.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:17 smithi131 ceph-mon[41654]: osdmap e5: 1 total, 0 up, 1 in 2024-02-05T07:46:18.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:17 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:46:18.654 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:18.341+0000 7f8d6c947200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T07:46:18.654 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:18.440+0000 7f8d6c947200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T07:46:18.654 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:46:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:46:18.652+0000 7f8d6c947200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T07:46:18.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:18.686+0000 7f55e2ffd700 1 -- 172.21.15.131:0/2093630350 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 14) v1 ==== 91316+0+0 (secure 0 0 0) 0x7f55dc01fc90 con 0x7f55ec105110 2024-02-05T07:46:18.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3707559753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:46:18.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:18 smithi157 ceph-mon[33345]: Standby manager daemon x started 2024-02-05T07:46:18.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:18 smithi157 ceph-mon[33345]: from='mgr.? 172.21.15.157:0/1690295077' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T07:46:18.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:18 smithi157 ceph-mon[33345]: from='mgr.? 172.21.15.157:0/1690295077' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T07:46:18.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:18 smithi157 ceph-mon[33345]: from='mgr.? 172.21.15.157:0/1690295077' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T07:46:18.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:18 smithi157 ceph-mon[33345]: from='mgr.? 172.21.15.157:0/1690295077' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T07:46:19.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3707559753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:46:19.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:18 smithi131 ceph-mon[29507]: Standby manager daemon x started 2024-02-05T07:46:19.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:18 smithi131 ceph-mon[29507]: from='mgr.? 172.21.15.157:0/1690295077' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T07:46:19.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:18 smithi131 ceph-mon[29507]: from='mgr.? 172.21.15.157:0/1690295077' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T07:46:19.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:18 smithi131 ceph-mon[29507]: from='mgr.? 172.21.15.157:0/1690295077' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T07:46:19.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:18 smithi131 ceph-mon[29507]: from='mgr.? 172.21.15.157:0/1690295077' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T07:46:19.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3707559753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:46:19.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:18 smithi131 ceph-mon[41654]: Standby manager daemon x started 2024-02-05T07:46:19.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:18 smithi131 ceph-mon[41654]: from='mgr.? 172.21.15.157:0/1690295077' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T07:46:19.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:18 smithi131 ceph-mon[41654]: from='mgr.? 172.21.15.157:0/1690295077' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T07:46:19.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:18 smithi131 ceph-mon[41654]: from='mgr.? 172.21.15.157:0/1690295077' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T07:46:19.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:18 smithi131 ceph-mon[41654]: from='mgr.? 172.21.15.157:0/1690295077' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T07:46:19.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:19 smithi157 ceph-mon[33345]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:19.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:19 smithi157 ceph-mon[33345]: mgrmap e14: y(active, since 68s), standbys: x 2024-02-05T07:46:19.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:19 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T07:46:20.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:19 smithi131 ceph-mon[29507]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:20.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:19 smithi131 ceph-mon[29507]: mgrmap e14: y(active, since 68s), standbys: x 2024-02-05T07:46:20.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:19 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T07:46:20.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:19 smithi131 ceph-mon[41654]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:20.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:19 smithi131 ceph-mon[41654]: mgrmap e14: y(active, since 68s), standbys: x 2024-02-05T07:46:20.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:19 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T07:46:21.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:21 smithi157 ceph-mon[33345]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:22.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:21 smithi131 ceph-mon[29507]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:22.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:21 smithi131 ceph-mon[41654]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:23.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:23 smithi157 ceph-mon[33345]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:24.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:23 smithi131 ceph-mon[29507]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:24.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:23 smithi131 ceph-mon[41654]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:24.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:24 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-05T07:46:24.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:24 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:25.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:24 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-05T07:46:25.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:24 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:25.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:24 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-05T07:46:25.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:24 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:25.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:25 smithi157 ceph-mon[33345]: Deploying daemon osd.0 on smithi131 2024-02-05T07:46:25.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:25 smithi157 ceph-mon[33345]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:26.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:25 smithi131 ceph-mon[41654]: Deploying daemon osd.0 on smithi131 2024-02-05T07:46:26.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:25 smithi131 ceph-mon[41654]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:26.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:25 smithi131 ceph-mon[29507]: Deploying daemon osd.0 on smithi131 2024-02-05T07:46:26.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:25 smithi131 ceph-mon[29507]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:27.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:27 smithi157 ceph-mon[33345]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:28.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:27 smithi131 ceph-mon[29507]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:28.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:27 smithi131 ceph-mon[41654]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:29.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:29 smithi157 ceph-mon[33345]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:30.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:29 smithi131 ceph-mon[29507]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:30.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:29 smithi131 ceph-mon[41654]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:30 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:30 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:30 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:30 smithi157 ceph-mon[33345]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:30 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:30 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:30 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:30 smithi131 ceph-mon[29507]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:30 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:30 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:30 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:30 smithi131 ceph-mon[41654]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:33.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:33.235+0000 7f55e2ffd700 1 -- 172.21.15.131:0/2093630350 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f55ec062160 con 0x7f55d8039180 2024-02-05T07:46:33.236 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 0 on host 'smithi131' 2024-02-05T07:46:33.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:33.237+0000 7f55f3b39700 1 -- 172.21.15.131:0/2093630350 >> v2:172.21.15.131:6800/760045032 conn(0x7f55d8039180 msgr2=0x7f55d803b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:33.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:33.237+0000 7f55f3b39700 1 --2- 172.21.15.131:0/2093630350 >> v2:172.21.15.131:6800/760045032 conn(0x7f55d8039180 0x7f55d803b640 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f55e8005950 tx=0x7f55e800b410 comp rx=0 tx=0).stop 2024-02-05T07:46:33.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:33.237+0000 7f55f3b39700 1 -- 172.21.15.131:0/2093630350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55ec105110 msgr2=0x7f55ec197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:33.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:33.237+0000 7f55f3b39700 1 --2- 172.21.15.131:0/2093630350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55ec105110 0x7f55ec197970 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009230 tx=0x7f55dc00b560 comp rx=0 tx=0).stop 2024-02-05T07:46:33.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:33.237+0000 7f55f3b39700 1 -- 172.21.15.131:0/2093630350 shutdown_connections 2024-02-05T07:46:33.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:33.237+0000 7f55f3b39700 1 --2- 172.21.15.131:0/2093630350 >> v2:172.21.15.131:6800/760045032 conn(0x7f55d8039180 0x7f55d803b640 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:33.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:33.237+0000 7f55f3b39700 1 --2- 172.21.15.131:0/2093630350 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f55ec107a40 0x7f55ec19e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:33.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:33.237+0000 7f55f3b39700 1 --2- 172.21.15.131:0/2093630350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55ec1027e0 0x7f55ec197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:33.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:33.237+0000 7f55f3b39700 1 --2- 172.21.15.131:0/2093630350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55ec105110 0x7f55ec197970 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:33.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:33.237+0000 7f55f3b39700 1 -- 172.21.15.131:0/2093630350 >> 172.21.15.131:0/2093630350 conn(0x7f55ec0fc210 msgr2=0x7f55ec103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:46:33.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:33.238+0000 7f55f3b39700 1 -- 172.21.15.131:0/2093630350 shutdown_connections 2024-02-05T07:46:33.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:33.238+0000 7f55f3b39700 1 -- 172.21.15.131:0/2093630350 wait complete. 2024-02-05T07:46:33.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:33 smithi131 ceph-mon[29507]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:33.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:33 smithi131 ceph-mon[29507]: from='osd.0 v2:172.21.15.131:6801/1614402360' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-05T07:46:33.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:33 smithi131 ceph-mon[29507]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-05T07:46:33.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:33.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:33.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:33 smithi131 ceph-mon[41654]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:33.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:33 smithi131 ceph-mon[41654]: from='osd.0 v2:172.21.15.131:6801/1614402360' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-05T07:46:33.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:33 smithi131 ceph-mon[41654]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-05T07:46:33.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:33.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:33.854 DEBUG:teuthology.orchestra.run.smithi131:osd.0> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.0.service 2024-02-05T07:46:33.857 INFO:tasks.cephadm:Deploying osd.1 on smithi131 with /dev/vg_nvme/lv_3... 2024-02-05T07:46:33.857 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-05T07:46:33.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:33 smithi157 ceph-mon[33345]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:33.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:33 smithi157 ceph-mon[33345]: from='osd.0 v2:172.21.15.131:6801/1614402360' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-05T07:46:33.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:33 smithi157 ceph-mon[33345]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-05T07:46:33.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:33.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:34.146 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:46:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:34 smithi157 ceph-mon[33345]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-05T07:46:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:34 smithi157 ceph-mon[33345]: osdmap e6: 1 total, 0 up, 1 in 2024-02-05T07:46:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:34 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:46:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:34 smithi157 ceph-mon[33345]: from='osd.0 v2:172.21.15.131:6801/1614402360' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-02-05T07:46:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:34 smithi157 ceph-mon[33345]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-02-05T07:46:34.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:34 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:34.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:34 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:34.992 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:34 smithi131 ceph-mon[41654]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-05T07:46:34.992 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:34 smithi131 ceph-mon[41654]: osdmap e6: 1 total, 0 up, 1 in 2024-02-05T07:46:34.992 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:34 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:46:34.992 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:34 smithi131 ceph-mon[41654]: from='osd.0 v2:172.21.15.131:6801/1614402360' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-02-05T07:46:34.992 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:34 smithi131 ceph-mon[41654]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-02-05T07:46:34.992 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:34 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:34.992 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:34 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:34.993 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:34 smithi131 ceph-mon[29507]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-05T07:46:34.993 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:34 smithi131 ceph-mon[29507]: osdmap e6: 1 total, 0 up, 1 in 2024-02-05T07:46:34.993 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:34 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:46:34.993 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:34 smithi131 ceph-mon[29507]: from='osd.0 v2:172.21.15.131:6801/1614402360' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-02-05T07:46:34.993 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:34 smithi131 ceph-mon[29507]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-02-05T07:46:34.993 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:34 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:34.993 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:34 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:34.993 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 07:46:34 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-0[46056]: 2024-02-05T07:46:34.698+0000 7fd25e861700 -1 osd.0 0 waiting for initial osdmap 2024-02-05T07:46:34.993 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 07:46:34 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-0[46056]: 2024-02-05T07:46:34.701+0000 7fd25a65b700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T07:46:35.874 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:35 smithi131 ceph-mon[29507]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:35.874 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:35 smithi131 ceph-mon[29507]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-02-05T07:46:35.874 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:35 smithi131 ceph-mon[29507]: osdmap e7: 1 total, 0 up, 1 in 2024-02-05T07:46:35.874 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:46:35.874 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:46:35.874 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:35 smithi131 ceph-mon[41654]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:35.875 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:35 smithi131 ceph-mon[41654]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-02-05T07:46:35.875 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:35 smithi131 ceph-mon[41654]: osdmap e7: 1 total, 0 up, 1 in 2024-02-05T07:46:35.875 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:35 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:46:35.875 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:35 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:46:35.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:35 smithi157 ceph-mon[33345]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:35.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:35 smithi157 ceph-mon[33345]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-02-05T07:46:35.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:35 smithi157 ceph-mon[33345]: osdmap e7: 1 total, 0 up, 1 in 2024-02-05T07:46:35.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:35 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:46:35.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:35 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:46:36.782 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-05T07:46:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-05T07:46:36.783 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-02-05T07:46:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-02-05T07:46:36.783 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121384 s, 864 MB/s 2024-02-05T07:46:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-02-05T07:46:36.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:36 smithi157 ceph-mon[33345]: purged_snaps scrub starts 2024-02-05T07:46:36.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:36 smithi157 ceph-mon[33345]: purged_snaps scrub ok 2024-02-05T07:46:36.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:36 smithi157 ceph-mon[33345]: osd.0 v2:172.21.15.131:6801/1614402360 boot 2024-02-05T07:46:36.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:36 smithi157 ceph-mon[33345]: osdmap e8: 1 total, 1 up, 1 in 2024-02-05T07:46:36.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:36 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:46:36.972 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:36 smithi131 ceph-mon[41654]: purged_snaps scrub starts 2024-02-05T07:46:36.972 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:36 smithi131 ceph-mon[41654]: purged_snaps scrub ok 2024-02-05T07:46:36.972 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:36 smithi131 ceph-mon[41654]: osd.0 v2:172.21.15.131:6801/1614402360 boot 2024-02-05T07:46:36.972 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:36 smithi131 ceph-mon[41654]: osdmap e8: 1 total, 1 up, 1 in 2024-02-05T07:46:36.972 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:36 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:46:36.975 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:36 smithi131 ceph-mon[29507]: purged_snaps scrub starts 2024-02-05T07:46:36.975 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:36 smithi131 ceph-mon[29507]: purged_snaps scrub ok 2024-02-05T07:46:36.975 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:36 smithi131 ceph-mon[29507]: osd.0 v2:172.21.15.131:6801/1614402360 boot 2024-02-05T07:46:36.975 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:36 smithi131 ceph-mon[29507]: osdmap e8: 1 total, 1 up, 1 in 2024-02-05T07:46:36.975 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:36 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:46:37.513 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi131:vg_nvme/lv_3 2024-02-05T07:46:37.844 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:46:37.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:37 smithi131 ceph-mon[29507]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:37.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:37 smithi131 ceph-mon[29507]: osdmap e9: 1 total, 1 up, 1 in 2024-02-05T07:46:37.962 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:37 smithi131 ceph-mon[41654]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:37.962 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:37 smithi131 ceph-mon[41654]: osdmap e9: 1 total, 1 up, 1 in 2024-02-05T07:46:37.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:37 smithi157 ceph-mon[33345]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:46:37.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:37 smithi157 ceph-mon[33345]: osdmap e9: 1 total, 1 up, 1 in 2024-02-05T07:46:39.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:39 smithi131 ceph-mon[29507]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:39.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:39 smithi131 ceph-mon[41654]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:39.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:39 smithi157 ceph-mon[33345]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:40.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.148+0000 7fcb664b9700 1 -- 172.21.15.131:0/3446271365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb60101900 msgr2=0x7fcb601091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:40.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.148+0000 7fcb664b9700 1 --2- 172.21.15.131:0/3446271365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb60101900 0x7fcb601091c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009f90 tx=0x7fcb5800b330 comp rx=0 tx=0).stop 2024-02-05T07:46:40.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.148+0000 7fcb664b9700 1 -- 172.21.15.131:0/3446271365 shutdown_connections 2024-02-05T07:46:40.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.148+0000 7fcb664b9700 1 --2- 172.21.15.131:0/3446271365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb60101900 0x7fcb601091c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:40.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.148+0000 7fcb664b9700 1 --2- 172.21.15.131:0/3446271365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb60102220 0x7fcb60102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:40.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.148+0000 7fcb664b9700 1 --2- 172.21.15.131:0/3446271365 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fcb60100f60 0x7fcb601013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:40.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.148+0000 7fcb664b9700 1 -- 172.21.15.131:0/3446271365 >> 172.21.15.131:0/3446271365 conn(0x7fcb600fb9b0 msgr2=0x7fcb600fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:46:40.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.148+0000 7fcb664b9700 1 -- 172.21.15.131:0/3446271365 shutdown_connections 2024-02-05T07:46:40.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.148+0000 7fcb664b9700 1 -- 172.21.15.131:0/3446271365 wait complete. 2024-02-05T07:46:40.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.148+0000 7fcb664b9700 1 Processor -- start 2024-02-05T07:46:40.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.148+0000 7fcb664b9700 1 -- start start 2024-02-05T07:46:40.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb664b9700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fcb60100f60 0x7fcb601963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:40.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb664b9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb60101900 0x7fcb60196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:40.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb664b9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb60102220 0x7fcb6019d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:40.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb664b9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6019df20 con 0x7fcb60102220 2024-02-05T07:46:40.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb664b9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6019e060 con 0x7fcb60101900 2024-02-05T07:46:40.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb664b9700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6019e1f0 con 0x7fcb60100f60 2024-02-05T07:46:40.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb5ffff700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fcb60100f60 0x7fcb601963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:40.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb5ffff700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fcb60100f60 0x7fcb601963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:51232/0 (socket says 172.21.15.131:51232) 2024-02-05T07:46:40.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb64a56700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb60102220 0x7fcb6019d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:40.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb5ffff700 1 -- 172.21.15.131:0/3315430094 learned_addr learned my addr 172.21.15.131:0/3315430094 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:46:40.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb5f7fe700 1 --2- 172.21.15.131:0/3315430094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb60101900 0x7fcb60196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:40.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb5ffff700 1 -- 172.21.15.131:0/3315430094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb60101900 msgr2=0x7fcb60196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:40.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb5ffff700 1 --2- 172.21.15.131:0/3315430094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb60101900 0x7fcb60196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:40.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb5ffff700 1 -- 172.21.15.131:0/3315430094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb60102220 msgr2=0x7fcb6019d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:40.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb5ffff700 1 --2- 172.21.15.131:0/3315430094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb60102220 0x7fcb6019d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:40.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb5ffff700 1 -- 172.21.15.131:0/3315430094 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540095f0 con 0x7fcb60100f60 2024-02-05T07:46:40.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb64a56700 1 --2- 172.21.15.131:0/3315430094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb60102220 0x7fcb6019d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:46:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb5ffff700 1 --2- 172.21.15.131:0/3315430094 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fcb60100f60 0x7fcb601963e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009920 tx=0x7fcb50009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:46:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb5d7fa700 1 -- 172.21.15.131:0/3315430094 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50012070 con 0x7fcb60100f60 2024-02-05T07:46:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb664b9700 1 -- 172.21.15.131:0/3315430094 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb58009c40 con 0x7fcb60100f60 2024-02-05T07:46:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb5d7fa700 1 -- 172.21.15.131:0/3315430094 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fcb500044d0 con 0x7fcb60100f60 2024-02-05T07:46:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb5d7fa700 1 -- 172.21.15.131:0/3315430094 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50003d50 con 0x7fcb60100f60 2024-02-05T07:46:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.149+0000 7fcb664b9700 1 -- 172.21.15.131:0/3315430094 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb60100220 con 0x7fcb60100f60 2024-02-05T07:46:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.150+0000 7fcb664b9700 1 -- 172.21.15.131:0/3315430094 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb600689e0 con 0x7fcb60100f60 2024-02-05T07:46:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.151+0000 7fcb5d7fa700 1 -- 172.21.15.131:0/3315430094 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 14) v1 ==== 91316+0+0 (secure 0 0 0) 0x7fcb50016030 con 0x7fcb60100f60 2024-02-05T07:46:40.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.151+0000 7fcb5d7fa700 1 --2- 172.21.15.131:0/3315430094 >> v2:172.21.15.131:6800/760045032 conn(0x7fcb4806d900 0x7fcb4806fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:46:40.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.151+0000 7fcb5d7fa700 1 -- 172.21.15.131:0/3315430094 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1541+0+0 (secure 0 0 0) 0x7fcb5008cfb0 con 0x7fcb60100f60 2024-02-05T07:46:40.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.151+0000 7fcb5f7fe700 1 --2- 172.21.15.131:0/3315430094 >> v2:172.21.15.131:6800/760045032 conn(0x7fcb4806d900 0x7fcb4806fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:46:40.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.152+0000 7fcb5f7fe700 1 --2- 172.21.15.131:0/3315430094 >> v2:172.21.15.131:6800/760045032 conn(0x7fcb4806d900 0x7fcb4806fdc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcb54005fd0 tx=0x7fcb54009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:46:40.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.156+0000 7fcb5d7fa700 1 -- 172.21.15.131:0/3315430094 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcb500c0d20 con 0x7fcb60100f60 2024-02-05T07:46:40.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:40.358+0000 7fcb664b9700 1 -- 172.21.15.131:0/3315430094 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb6010bdc0 con 0x7fcb4806d900 2024-02-05T07:46:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: Detected new or changed devices on smithi131 2024-02-05T07:46:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:46:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: Adjusting osd_memory_target on smithi131 to 16064M 2024-02-05T07:46:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:46:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:46:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:46:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:41.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:41.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:41.229 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: Detected new or changed devices on smithi131 2024-02-05T07:46:41.229 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:41.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:41.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:46:41.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: Adjusting osd_memory_target on smithi131 to 16064M 2024-02-05T07:46:41.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:41.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:41.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:41.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:41.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:41.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:46:41.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:46:41.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:46:41.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:41.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:41.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:41.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:41.231 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:41.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: Detected new or changed devices on smithi131 2024-02-05T07:46:41.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:41.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:41.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:46:41.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: Adjusting osd_memory_target on smithi131 to 16064M 2024-02-05T07:46:41.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:41.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:41.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:41.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:41.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:41.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:46:41.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:46:41.232 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:46:41.232 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:41.232 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:41.232 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:41.232 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:41.232 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:43.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:43 smithi131 ceph-mon[29507]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:43.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1667404789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6068c48-3bc2-4728-9924-66bafeff3703"}]: dispatch 2024-02-05T07:46:43.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1667404789' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6068c48-3bc2-4728-9924-66bafeff3703"}]': finished 2024-02-05T07:46:43.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:43 smithi131 ceph-mon[29507]: osdmap e10: 2 total, 1 up, 2 in 2024-02-05T07:46:43.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:46:43.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:43 smithi131 ceph-mon[41654]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:43.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1667404789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6068c48-3bc2-4728-9924-66bafeff3703"}]: dispatch 2024-02-05T07:46:43.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1667404789' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6068c48-3bc2-4728-9924-66bafeff3703"}]': finished 2024-02-05T07:46:43.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:43 smithi131 ceph-mon[41654]: osdmap e10: 2 total, 1 up, 2 in 2024-02-05T07:46:43.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:43 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:46:43.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:43 smithi157 ceph-mon[33345]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:43.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1667404789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6068c48-3bc2-4728-9924-66bafeff3703"}]: dispatch 2024-02-05T07:46:43.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1667404789' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6068c48-3bc2-4728-9924-66bafeff3703"}]': finished 2024-02-05T07:46:43.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:43 smithi157 ceph-mon[33345]: osdmap e10: 2 total, 1 up, 2 in 2024-02-05T07:46:43.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:43 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:46:44.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/148019742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:46:45.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/148019742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:46:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/148019742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:46:45.964 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:45 smithi131 ceph-mon[29507]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:45.965 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:45 smithi131 ceph-mon[41654]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:45.970 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:45 smithi157 ceph-mon[33345]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:47.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:47 smithi131 ceph-mon[29507]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:47 smithi131 ceph-mon[41654]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:47.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:47 smithi157 ceph-mon[33345]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:49 smithi157 ceph-mon[33345]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:50.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:49 smithi131 ceph-mon[29507]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:49 smithi131 ceph-mon[41654]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:50 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-05T07:46:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:50 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:51.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:50 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-05T07:46:51.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:50 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:50 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-05T07:46:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:50 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:51.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:51 smithi157 ceph-mon[33345]: Deploying daemon osd.1 on smithi131 2024-02-05T07:46:51.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:51 smithi157 ceph-mon[33345]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:52.012 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:51 smithi131 ceph-mon[29507]: Deploying daemon osd.1 on smithi131 2024-02-05T07:46:52.012 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:51 smithi131 ceph-mon[29507]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:52.012 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:51 smithi131 ceph-mon[41654]: Deploying daemon osd.1 on smithi131 2024-02-05T07:46:52.012 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:51 smithi131 ceph-mon[41654]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:53.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:53 smithi157 ceph-mon[33345]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:53.981 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:53 smithi131 ceph-mon[29507]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:53.981 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:53 smithi131 ceph-mon[41654]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:55.951 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:55 smithi131 ceph-mon[29507]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:55.952 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:55.952 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:55.952 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:55.952 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:55 smithi131 ceph-mon[41654]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:55.952 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:55.952 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:55.952 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:55.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:55 smithi157 ceph-mon[33345]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:55.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:55.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:55.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:57.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:57 smithi131 ceph-mon[41654]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:57.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:57 smithi131 ceph-mon[29507]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:57.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:57 smithi157 ceph-mon[33345]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:58.910+0000 7fcb5d7fa700 1 -- 172.21.15.131:0/3315430094 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcb6010bdc0 con 0x7fcb4806d900 2024-02-05T07:46:58.911 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 1 on host 'smithi131' 2024-02-05T07:46:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:58.912+0000 7fcb664b9700 1 -- 172.21.15.131:0/3315430094 >> v2:172.21.15.131:6800/760045032 conn(0x7fcb4806d900 msgr2=0x7fcb4806fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:58.912+0000 7fcb664b9700 1 --2- 172.21.15.131:0/3315430094 >> v2:172.21.15.131:6800/760045032 conn(0x7fcb4806d900 0x7fcb4806fdc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcb54005fd0 tx=0x7fcb54009310 comp rx=0 tx=0).stop 2024-02-05T07:46:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:58.912+0000 7fcb664b9700 1 -- 172.21.15.131:0/3315430094 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fcb60100f60 msgr2=0x7fcb601963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:46:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:58.912+0000 7fcb664b9700 1 --2- 172.21.15.131:0/3315430094 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fcb60100f60 0x7fcb601963e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009920 tx=0x7fcb50009c30 comp rx=0 tx=0).stop 2024-02-05T07:46:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:58.912+0000 7fcb664b9700 1 -- 172.21.15.131:0/3315430094 shutdown_connections 2024-02-05T07:46:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:58.912+0000 7fcb664b9700 1 --2- 172.21.15.131:0/3315430094 >> v2:172.21.15.131:6800/760045032 conn(0x7fcb4806d900 0x7fcb4806fdc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:58.912+0000 7fcb664b9700 1 --2- 172.21.15.131:0/3315430094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb60102220 0x7fcb6019d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:58.912+0000 7fcb664b9700 1 --2- 172.21.15.131:0/3315430094 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fcb60100f60 0x7fcb601963e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:58.912+0000 7fcb664b9700 1 --2- 172.21.15.131:0/3315430094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb60101900 0x7fcb60196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:46:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:58.912+0000 7fcb664b9700 1 -- 172.21.15.131:0/3315430094 >> 172.21.15.131:0/3315430094 conn(0x7fcb600fb9b0 msgr2=0x7fcb600fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:46:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:58.912+0000 7fcb664b9700 1 -- 172.21.15.131:0/3315430094 shutdown_connections 2024-02-05T07:46:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:46:58.912+0000 7fcb664b9700 1 -- 172.21.15.131:0/3315430094 wait complete. 2024-02-05T07:46:59.203 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:59.203 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:59.203 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:59.203 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:59.203 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:59.203 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[41654]: from='osd.1 v2:172.21.15.131:6805/1256644882' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-05T07:46:59.203 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[41654]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:59.204 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:59.204 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:59.204 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:59.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:59.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:59.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:59.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:59.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:59.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[29507]: from='osd.1 v2:172.21.15.131:6805/1256644882' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-05T07:46:59.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[29507]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:59.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:59.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:59.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:46:59 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:59.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:59 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:59.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:59 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:59.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:59 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:59.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:59 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:59.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:59 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:59.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:59 smithi157 ceph-mon[33345]: from='osd.1 v2:172.21.15.131:6805/1256644882' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-05T07:46:59.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:59 smithi157 ceph-mon[33345]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:46:59.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:59 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:59.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:59 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:46:59.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:46:59 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:59.478 DEBUG:teuthology.orchestra.run.smithi131:osd.1> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.1.service 2024-02-05T07:46:59.480 INFO:tasks.cephadm:Deploying osd.2 on smithi131 with /dev/vg_nvme/lv_2... 2024-02-05T07:46:59.480 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-05T07:47:00.251 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:47:00.373 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:00 smithi131 ceph-mon[29507]: from='osd.1 v2:172.21.15.131:6805/1256644882' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-05T07:47:00.373 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:00 smithi131 ceph-mon[29507]: osdmap e11: 2 total, 1 up, 2 in 2024-02-05T07:47:00.373 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:00 smithi131 ceph-mon[29507]: from='osd.1 v2:172.21.15.131:6805/1256644882' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-02-05T07:47:00.373 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:00 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:47:00.374 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:00 smithi131 ceph-mon[41654]: from='osd.1 v2:172.21.15.131:6805/1256644882' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-05T07:47:00.374 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:00 smithi131 ceph-mon[41654]: osdmap e11: 2 total, 1 up, 2 in 2024-02-05T07:47:00.374 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:00 smithi131 ceph-mon[41654]: from='osd.1 v2:172.21.15.131:6805/1256644882' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-02-05T07:47:00.374 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:00 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:47:00.374 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 07:47:00 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-1[52280]: 2024-02-05T07:47:00.053+0000 7fe27aea7700 -1 osd.1 0 waiting for initial osdmap 2024-02-05T07:47:00.374 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 07:47:00 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-1[52280]: 2024-02-05T07:47:00.060+0000 7fe276ca1700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T07:47:00.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:00 smithi157 ceph-mon[33345]: from='osd.1 v2:172.21.15.131:6805/1256644882' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-05T07:47:00.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:00 smithi157 ceph-mon[33345]: osdmap e11: 2 total, 1 up, 2 in 2024-02-05T07:47:00.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:00 smithi157 ceph-mon[33345]: from='osd.1 v2:172.21.15.131:6805/1256644882' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-02-05T07:47:00.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:00 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:47:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:01 smithi131 ceph-mon[29507]: from='osd.1 v2:172.21.15.131:6805/1256644882' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-02-05T07:47:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:01 smithi131 ceph-mon[29507]: osdmap e12: 2 total, 1 up, 2 in 2024-02-05T07:47:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:47:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:47:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:01 smithi131 ceph-mon[29507]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:47:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:01 smithi131 ceph-mon[41654]: from='osd.1 v2:172.21.15.131:6805/1256644882' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-02-05T07:47:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:01 smithi131 ceph-mon[41654]: osdmap e12: 2 total, 1 up, 2 in 2024-02-05T07:47:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:47:01.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:47:01.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:01 smithi131 ceph-mon[41654]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:47:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:01 smithi157 ceph-mon[33345]: from='osd.1 v2:172.21.15.131:6805/1256644882' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-02-05T07:47:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:01 smithi157 ceph-mon[33345]: osdmap e12: 2 total, 1 up, 2 in 2024-02-05T07:47:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:47:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:47:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:01 smithi157 ceph-mon[33345]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:47:02.365 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:02 smithi131 ceph-mon[29507]: purged_snaps scrub starts 2024-02-05T07:47:02.365 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:02 smithi131 ceph-mon[29507]: purged_snaps scrub ok 2024-02-05T07:47:02.365 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:02 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:47:02.365 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:02 smithi131 ceph-mon[29507]: osd.1 v2:172.21.15.131:6805/1256644882 boot 2024-02-05T07:47:02.366 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:02 smithi131 ceph-mon[29507]: osdmap e13: 2 total, 2 up, 2 in 2024-02-05T07:47:02.366 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:02 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:47:02.366 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:02 smithi131 ceph-mon[41654]: purged_snaps scrub starts 2024-02-05T07:47:02.366 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:02 smithi131 ceph-mon[41654]: purged_snaps scrub ok 2024-02-05T07:47:02.366 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:02 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:47:02.366 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:02 smithi131 ceph-mon[41654]: osd.1 v2:172.21.15.131:6805/1256644882 boot 2024-02-05T07:47:02.366 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:02 smithi131 ceph-mon[41654]: osdmap e13: 2 total, 2 up, 2 in 2024-02-05T07:47:02.366 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:02 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:47:02.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:02 smithi157 ceph-mon[33345]: purged_snaps scrub starts 2024-02-05T07:47:02.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:02 smithi157 ceph-mon[33345]: purged_snaps scrub ok 2024-02-05T07:47:02.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:02 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:47:02.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:02 smithi157 ceph-mon[33345]: osd.1 v2:172.21.15.131:6805/1256644882 boot 2024-02-05T07:47:02.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:02 smithi157 ceph-mon[33345]: osdmap e13: 2 total, 2 up, 2 in 2024-02-05T07:47:02.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:02 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:47:03.028 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-05T07:47:03.029 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-05T07:47:03.029 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-02-05T07:47:03.029 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-02-05T07:47:03.029 INFO:teuthology.orchestra.run.smithi131.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0134345 s, 781 MB/s 2024-02-05T07:47:03.029 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-02-05T07:47:03.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:03 smithi131 ceph-mon[41654]: osdmap e14: 2 total, 2 up, 2 in 2024-02-05T07:47:03.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:03 smithi131 ceph-mon[41654]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:03.363 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:03 smithi131 ceph-mon[29507]: osdmap e14: 2 total, 2 up, 2 in 2024-02-05T07:47:03.363 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:03 smithi131 ceph-mon[29507]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:03 smithi157 ceph-mon[33345]: osdmap e14: 2 total, 2 up, 2 in 2024-02-05T07:47:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:03 smithi157 ceph-mon[33345]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:03.732 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi131:vg_nvme/lv_2 2024-02-05T07:47:04.409 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:47:05.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[41654]: Detected new or changed devices on smithi131 2024-02-05T07:47:05.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:05.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:05.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:05.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:05.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[41654]: Adjusting osd_memory_target on smithi131 to 8032M 2024-02-05T07:47:05.548 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:05.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:05.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:05.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:05.549 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[41654]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:05.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[29507]: Detected new or changed devices on smithi131 2024-02-05T07:47:05.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:05.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:05.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:05.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:05.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[29507]: Adjusting osd_memory_target on smithi131 to 8032M 2024-02-05T07:47:05.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:05.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:05.549 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:05.550 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:05.550 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:05 smithi131 ceph-mon[29507]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:05 smithi157 ceph-mon[33345]: Detected new or changed devices on smithi131 2024-02-05T07:47:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:05.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:05 smithi157 ceph-mon[33345]: Adjusting osd_memory_target on smithi131 to 8032M 2024-02-05T07:47:05.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:05.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:05.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:05.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:05.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:05 smithi157 ceph-mon[33345]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:06.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.048+0000 7ff55fc3f700 1 -- 172.21.15.131:0/662660550 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff558101e40 msgr2=0x7ff5581022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:47:06.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.048+0000 7ff55fc3f700 1 --2- 172.21.15.131:0/662660550 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff558101e40 0x7ff5581022a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff548009990 tx=0x7ff548009ca0 comp rx=0 tx=0).stop 2024-02-05T07:47:06.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.048+0000 7ff55fc3f700 1 -- 172.21.15.131:0/662660550 shutdown_connections 2024-02-05T07:47:06.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.048+0000 7ff55fc3f700 1 --2- 172.21.15.131:0/662660550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff558100c40 0x7ff558101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:06.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.048+0000 7ff55fc3f700 1 --2- 172.21.15.131:0/662660550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5581027e0 0x7ff558109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:06.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.048+0000 7ff55fc3f700 1 --2- 172.21.15.131:0/662660550 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff558101e40 0x7ff5581022a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:06.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.048+0000 7ff55fc3f700 1 -- 172.21.15.131:0/662660550 >> 172.21.15.131:0/662660550 conn(0x7ff5580fc220 msgr2=0x7ff5580fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:47:06.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.048+0000 7ff55fc3f700 1 -- 172.21.15.131:0/662660550 shutdown_connections 2024-02-05T07:47:06.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.048+0000 7ff55fc3f700 1 -- 172.21.15.131:0/662660550 wait complete. 2024-02-05T07:47:06.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55fc3f700 1 Processor -- start 2024-02-05T07:47:06.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55fc3f700 1 -- start start 2024-02-05T07:47:06.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55fc3f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff558100c40 0x7ff558196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:47:06.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55fc3f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff558101e40 0x7ff5581967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:47:06.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55fc3f700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff5581027e0 0x7ff55819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:47:06.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55fc3f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55819de00 con 0x7ff558100c40 2024-02-05T07:47:06.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55fc3f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55819df40 con 0x7ff558101e40 2024-02-05T07:47:06.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55fc3f700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55819e0d0 con 0x7ff5581027e0 2024-02-05T07:47:06.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55d9db700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff558100c40 0x7ff558196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:47:06.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55d9db700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff558100c40 0x7ff558196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59792/0 (socket says 172.21.15.131:59792) 2024-02-05T07:47:06.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55e1dc700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff5581027e0 0x7ff55819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:47:06.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55d9db700 1 -- 172.21.15.131:0/3983702416 learned_addr learned my addr 172.21.15.131:0/3983702416 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:47:06.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55d1da700 1 --2- 172.21.15.131:0/3983702416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff558101e40 0x7ff5581967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:47:06.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55d9db700 1 -- 172.21.15.131:0/3983702416 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff5581027e0 msgr2=0x7ff55819d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:47:06.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55d9db700 1 --2- 172.21.15.131:0/3983702416 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff5581027e0 0x7ff55819d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:06.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55d9db700 1 -- 172.21.15.131:0/3983702416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff558101e40 msgr2=0x7ff5581967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:47:06.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55d9db700 1 --2- 172.21.15.131:0/3983702416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff558101e40 0x7ff5581967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:06.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55d9db700 1 -- 172.21.15.131:0/3983702416 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff554009520 con 0x7ff558100c40 2024-02-05T07:47:06.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55e1dc700 1 --2- 172.21.15.131:0/3983702416 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff5581027e0 0x7ff55819d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:47:06.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.049+0000 7ff55d9db700 1 --2- 172.21.15.131:0/3983702416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff558100c40 0x7ff558196270 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff55400ea80 tx=0x7ff55400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:47:06.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.050+0000 7ff54effd700 1 -- 172.21.15.131:0/3983702416 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55400cbd0 con 0x7ff558100c40 2024-02-05T07:47:06.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.050+0000 7ff55fc3f700 1 -- 172.21.15.131:0/3983702416 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5480095f0 con 0x7ff558100c40 2024-02-05T07:47:06.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.050+0000 7ff54effd700 1 -- 172.21.15.131:0/3983702416 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7ff55400cd30 con 0x7ff558100c40 2024-02-05T07:47:06.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.050+0000 7ff54effd700 1 -- 172.21.15.131:0/3983702416 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554018680 con 0x7ff558100c40 2024-02-05T07:47:06.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.050+0000 7ff55fc3f700 1 -- 172.21.15.131:0/3983702416 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55819e770 con 0x7ff558100c40 2024-02-05T07:47:06.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.051+0000 7ff55fc3f700 1 -- 172.21.15.131:0/3983702416 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff558107820 con 0x7ff558100c40 2024-02-05T07:47:06.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.051+0000 7ff54effd700 1 -- 172.21.15.131:0/3983702416 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 14) v1 ==== 91316+0+0 (secure 0 0 0) 0x7ff554010430 con 0x7ff558100c40 2024-02-05T07:47:06.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.052+0000 7ff54effd700 1 --2- 172.21.15.131:0/3983702416 >> v2:172.21.15.131:6800/760045032 conn(0x7ff54406d900 0x7ff54406fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:47:06.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.052+0000 7ff54effd700 1 -- 172.21.15.131:0/3983702416 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 1833+0+0 (secure 0 0 0) 0x7ff554014070 con 0x7ff558100c40 2024-02-05T07:47:06.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.052+0000 7ff55d1da700 1 --2- 172.21.15.131:0/3983702416 >> v2:172.21.15.131:6800/760045032 conn(0x7ff54406d900 0x7ff54406fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:47:06.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.052+0000 7ff55d1da700 1 --2- 172.21.15.131:0/3983702416 >> v2:172.21.15.131:6800/760045032 conn(0x7ff54406d900 0x7ff54406fdc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff54800b2e0 tx=0x7ff54800b6f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:47:06.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.056+0000 7ff54effd700 1 -- 172.21.15.131:0/3983702416 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff554056f50 con 0x7ff558100c40 2024-02-05T07:47:06.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:06.259+0000 7ff55fc3f700 1 -- 172.21.15.131:0/3983702416 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff55810be50 con 0x7ff54406d900 2024-02-05T07:47:06.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:06 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:47:06.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:06 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:47:06.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:06 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:06.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:06 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:47:06.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:06 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:47:06.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:06 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:06.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:06 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:47:06.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:06 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:47:06.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:06 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:07.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:07 smithi131 ceph-mon[29507]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:47:07.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:07 smithi131 ceph-mon[29507]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:07.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:07 smithi131 ceph-mon[41654]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:47:07.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:07 smithi131 ceph-mon[41654]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:07.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:07 smithi157 ceph-mon[33345]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:47:07.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:07 smithi157 ceph-mon[33345]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:09.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:09 smithi157 ceph-mon[33345]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:09.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/278218743' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d0cfb98-091c-467e-abe5-abe3b8542323"}]: dispatch 2024-02-05T07:47:09.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d0cfb98-091c-467e-abe5-abe3b8542323"}]: dispatch 2024-02-05T07:47:09.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d0cfb98-091c-467e-abe5-abe3b8542323"}]': finished 2024-02-05T07:47:09.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:09 smithi157 ceph-mon[33345]: osdmap e15: 3 total, 2 up, 3 in 2024-02-05T07:47:09.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:09 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:09.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3098763983' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:47:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:09 smithi131 ceph-mon[41654]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/278218743' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d0cfb98-091c-467e-abe5-abe3b8542323"}]: dispatch 2024-02-05T07:47:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d0cfb98-091c-467e-abe5-abe3b8542323"}]: dispatch 2024-02-05T07:47:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d0cfb98-091c-467e-abe5-abe3b8542323"}]': finished 2024-02-05T07:47:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:09 smithi131 ceph-mon[41654]: osdmap e15: 3 total, 2 up, 3 in 2024-02-05T07:47:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:09 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3098763983' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:47:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:09 smithi131 ceph-mon[29507]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/278218743' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d0cfb98-091c-467e-abe5-abe3b8542323"}]: dispatch 2024-02-05T07:47:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d0cfb98-091c-467e-abe5-abe3b8542323"}]: dispatch 2024-02-05T07:47:10.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d0cfb98-091c-467e-abe5-abe3b8542323"}]': finished 2024-02-05T07:47:10.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:09 smithi131 ceph-mon[29507]: osdmap e15: 3 total, 2 up, 3 in 2024-02-05T07:47:10.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:09 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:10.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3098763983' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:47:11.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:11 smithi157 ceph-mon[33345]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:12.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:11 smithi131 ceph-mon[29507]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:12.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:11 smithi131 ceph-mon[41654]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:13.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:13 smithi157 ceph-mon[33345]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:13 smithi131 ceph-mon[41654]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:14.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:13 smithi131 ceph-mon[29507]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:15.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:15 smithi131 ceph-mon[41654]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:15.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:15 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-05T07:47:15.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:15 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:15.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:15 smithi131 ceph-mon[29507]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:15.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:15 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-05T07:47:15.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:15 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:15.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:15 smithi157 ceph-mon[33345]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:15.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:15 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-05T07:47:15.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:15 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:16.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:16 smithi157 ceph-mon[33345]: Deploying daemon osd.2 on smithi131 2024-02-05T07:47:17.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:16 smithi131 ceph-mon[29507]: Deploying daemon osd.2 on smithi131 2024-02-05T07:47:17.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:16 smithi131 ceph-mon[41654]: Deploying daemon osd.2 on smithi131 2024-02-05T07:47:17.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:17 smithi157 ceph-mon[33345]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:18.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:17 smithi131 ceph-mon[29507]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:18.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:17 smithi131 ceph-mon[41654]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:19.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:19 smithi157 ceph-mon[33345]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:19.980 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:19 smithi131 ceph-mon[29507]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:19.981 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:19 smithi131 ceph-mon[41654]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:21.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:21 smithi131 ceph-mon[29507]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:21.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:21 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:21.758 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:21 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:21.759 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:21 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:47:21.759 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:21 smithi131 ceph-mon[41654]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:21.759 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:21 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:21.759 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:21 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:21.759 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:21 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:47:21.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:21 smithi157 ceph-mon[33345]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:21.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:21 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:21.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:21 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:21.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:21 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:47:23.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:23 smithi157 ceph-mon[33345]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:24.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:23 smithi131 ceph-mon[29507]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:24.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:23 smithi131 ceph-mon[41654]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:24.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:24.811+0000 7ff54effd700 1 -- 172.21.15.131:0/3983702416 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff55810be50 con 0x7ff54406d900 2024-02-05T07:47:24.813 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 2 on host 'smithi131' 2024-02-05T07:47:24.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:24.813+0000 7ff55fc3f700 1 -- 172.21.15.131:0/3983702416 >> v2:172.21.15.131:6800/760045032 conn(0x7ff54406d900 msgr2=0x7ff54406fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:47:24.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:24.813+0000 7ff55fc3f700 1 --2- 172.21.15.131:0/3983702416 >> v2:172.21.15.131:6800/760045032 conn(0x7ff54406d900 0x7ff54406fdc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff54800b2e0 tx=0x7ff54800b6f0 comp rx=0 tx=0).stop 2024-02-05T07:47:24.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:24.813+0000 7ff55fc3f700 1 -- 172.21.15.131:0/3983702416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff558100c40 msgr2=0x7ff558196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:47:24.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:24.813+0000 7ff55fc3f700 1 --2- 172.21.15.131:0/3983702416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff558100c40 0x7ff558196270 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff55400ea80 tx=0x7ff55400ed90 comp rx=0 tx=0).stop 2024-02-05T07:47:24.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:24.814+0000 7ff55fc3f700 1 -- 172.21.15.131:0/3983702416 shutdown_connections 2024-02-05T07:47:24.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:24.814+0000 7ff55fc3f700 1 --2- 172.21.15.131:0/3983702416 >> v2:172.21.15.131:6800/760045032 conn(0x7ff54406d900 0x7ff54406fdc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:24.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:24.814+0000 7ff55fc3f700 1 --2- 172.21.15.131:0/3983702416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff558100c40 0x7ff558196270 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:24.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:24.814+0000 7ff55fc3f700 1 --2- 172.21.15.131:0/3983702416 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff5581027e0 0x7ff55819d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:24.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:24.814+0000 7ff55fc3f700 1 --2- 172.21.15.131:0/3983702416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff558101e40 0x7ff5581967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:24.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:24.814+0000 7ff55fc3f700 1 -- 172.21.15.131:0/3983702416 >> 172.21.15.131:0/3983702416 conn(0x7ff5580fc220 msgr2=0x7ff5580fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:47:24.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:24.814+0000 7ff55fc3f700 1 -- 172.21.15.131:0/3983702416 shutdown_connections 2024-02-05T07:47:24.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:24.814+0000 7ff55fc3f700 1 -- 172.21.15.131:0/3983702416 wait complete. 2024-02-05T07:47:25.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:24 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:25.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:24 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:25.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:24 smithi131 ceph-mon[29507]: from='osd.2 v2:172.21.15.131:6809/1463881636' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-05T07:47:25.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:24 smithi131 ceph-mon[29507]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:25.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:24 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:25.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:24 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:25.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:24 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:25.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:24 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:25.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:24 smithi131 ceph-mon[41654]: from='osd.2 v2:172.21.15.131:6809/1463881636' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-05T07:47:25.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:24 smithi131 ceph-mon[41654]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:25.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:24 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:25.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:24 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:25.406 DEBUG:teuthology.orchestra.run.smithi131:osd.2> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.2.service 2024-02-05T07:47:25.408 INFO:tasks.cephadm:Deploying osd.3 on smithi131 with /dev/vg_nvme/lv_1... 2024-02-05T07:47:25.408 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-05T07:47:25.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:24 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:25.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:24 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:25.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:24 smithi157 ceph-mon[33345]: from='osd.2 v2:172.21.15.131:6809/1463881636' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-05T07:47:25.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:24 smithi157 ceph-mon[33345]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:25.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:24 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:25.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:24 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:25.691 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:47:26.066 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:26 smithi131 ceph-mon[41654]: from='osd.2 v2:172.21.15.131:6809/1463881636' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-05T07:47:26.067 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:26 smithi131 ceph-mon[41654]: osdmap e16: 3 total, 2 up, 3 in 2024-02-05T07:47:26.067 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:26 smithi131 ceph-mon[41654]: from='osd.2 v2:172.21.15.131:6809/1463881636' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-02-05T07:47:26.067 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:26 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:26.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:26 smithi131 ceph-mon[29507]: from='osd.2 v2:172.21.15.131:6809/1463881636' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-05T07:47:26.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:26 smithi131 ceph-mon[29507]: osdmap e16: 3 total, 2 up, 3 in 2024-02-05T07:47:26.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:26 smithi131 ceph-mon[29507]: from='osd.2 v2:172.21.15.131:6809/1463881636' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-02-05T07:47:26.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:26 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:26.068 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 07:47:26 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2[58548]: 2024-02-05T07:47:26.003+0000 7f9766d0b700 -1 osd.2 0 waiting for initial osdmap 2024-02-05T07:47:26.068 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 07:47:26 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2[58548]: 2024-02-05T07:47:26.011+0000 7f9762b05700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T07:47:26.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:26 smithi157 ceph-mon[33345]: from='osd.2 v2:172.21.15.131:6809/1463881636' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-05T07:47:26.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:26 smithi157 ceph-mon[33345]: osdmap e16: 3 total, 2 up, 3 in 2024-02-05T07:47:26.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:26 smithi157 ceph-mon[33345]: from='osd.2 v2:172.21.15.131:6809/1463881636' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-02-05T07:47:26.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:26 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:27.176 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:27 smithi131 ceph-mon[41654]: from='osd.2 v2:172.21.15.131:6809/1463881636' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-02-05T07:47:27.176 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:27 smithi131 ceph-mon[41654]: osdmap e17: 3 total, 2 up, 3 in 2024-02-05T07:47:27.176 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:27 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:27.176 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:27 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:27.176 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:27 smithi131 ceph-mon[41654]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:27.177 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:27 smithi131 ceph-mon[29507]: from='osd.2 v2:172.21.15.131:6809/1463881636' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-02-05T07:47:27.177 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:27 smithi131 ceph-mon[29507]: osdmap e17: 3 total, 2 up, 3 in 2024-02-05T07:47:27.177 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:27 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:27.177 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:27 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:27.177 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:27 smithi131 ceph-mon[29507]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:27.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:27 smithi157 ceph-mon[33345]: from='osd.2 v2:172.21.15.131:6809/1463881636' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-02-05T07:47:27.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:27 smithi157 ceph-mon[33345]: osdmap e17: 3 total, 2 up, 3 in 2024-02-05T07:47:27.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:27 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:27.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:27 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:27.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:27 smithi157 ceph-mon[33345]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:47:28.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:28 smithi131 ceph-mon[41654]: purged_snaps scrub starts 2024-02-05T07:47:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:28 smithi131 ceph-mon[41654]: purged_snaps scrub ok 2024-02-05T07:47:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:28 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:28 smithi131 ceph-mon[41654]: osd.2 v2:172.21.15.131:6809/1463881636 boot 2024-02-05T07:47:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:28 smithi131 ceph-mon[41654]: osdmap e18: 3 total, 3 up, 3 in 2024-02-05T07:47:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:28 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:28.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:28 smithi131 ceph-mon[29507]: purged_snaps scrub starts 2024-02-05T07:47:28.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:28 smithi131 ceph-mon[29507]: purged_snaps scrub ok 2024-02-05T07:47:28.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:28 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:28.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:28 smithi131 ceph-mon[29507]: osd.2 v2:172.21.15.131:6809/1463881636 boot 2024-02-05T07:47:28.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:28 smithi131 ceph-mon[29507]: osdmap e18: 3 total, 3 up, 3 in 2024-02-05T07:47:28.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:28 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:28 smithi157 ceph-mon[33345]: purged_snaps scrub starts 2024-02-05T07:47:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:28 smithi157 ceph-mon[33345]: purged_snaps scrub ok 2024-02-05T07:47:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:28 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:28 smithi157 ceph-mon[33345]: osd.2 v2:172.21.15.131:6809/1463881636 boot 2024-02-05T07:47:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:28 smithi157 ceph-mon[33345]: osdmap e18: 3 total, 3 up, 3 in 2024-02-05T07:47:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:28 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:47:28.585 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-05T07:47:28.585 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-05T07:47:28.585 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-02-05T07:47:28.585 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-02-05T07:47:28.585 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120291 s, 872 MB/s 2024-02-05T07:47:28.585 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-02-05T07:47:29.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:29 smithi131 ceph-mon[29507]: osdmap e19: 3 total, 3 up, 3 in 2024-02-05T07:47:29.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:29 smithi131 ceph-mon[29507]: pgmap v78: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:29.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:29 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:29 smithi131 ceph-mon[41654]: osdmap e19: 3 total, 3 up, 3 in 2024-02-05T07:47:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:29 smithi131 ceph-mon[41654]: pgmap v78: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:29 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:29.422 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi131:vg_nvme/lv_1 2024-02-05T07:47:29.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:29 smithi157 ceph-mon[33345]: osdmap e19: 3 total, 3 up, 3 in 2024-02-05T07:47:29.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:29 smithi157 ceph-mon[33345]: pgmap v78: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:29.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:29 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:30.092 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:47:30.219 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:30 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:30.219 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:30 smithi131 ceph-mon[41654]: osdmap e20: 3 total, 3 up, 3 in 2024-02-05T07:47:30.219 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:30 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:30.220 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:30 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:30.220 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:30 smithi131 ceph-mon[29507]: osdmap e20: 3 total, 3 up, 3 in 2024-02-05T07:47:30.220 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:30 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:30.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:30 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:30.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:30 smithi157 ceph-mon[33345]: osdmap e20: 3 total, 3 up, 3 in 2024-02-05T07:47:30.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:30 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:30.517 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 07:47:30 smithi131 sudo[62955]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-05T07:47:30.517 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 07:47:30 smithi131 sudo[62955]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:47:30.518 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 07:47:30 smithi131 sudo[62955]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:47:30.518 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 07:47:30 smithi131 sudo[62955]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:47:30.918 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 07:47:30 smithi131 sudo[62983]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-05T07:47:30.918 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 07:47:30 smithi131 sudo[62983]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:47:30.918 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 07:47:30 smithi131 sudo[62983]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:47:30.918 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 07:47:30 smithi131 sudo[62983]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:47:31.179 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 07:47:30 smithi131 sudo[62989]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-05T07:47:31.179 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 07:47:30 smithi131 sudo[62989]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:47:31.179 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 07:47:30 smithi131 sudo[62989]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:47:31.180 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 07:47:30 smithi131 sudo[62989]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:47:31.180 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: Detected new or changed devices on smithi131 2024-02-05T07:47:31.180 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.180 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:31.180 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: osdmap e21: 3 total, 3 up, 3 in 2024-02-05T07:47:31.180 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.180 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:31.180 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:31.180 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:31.180 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: Adjusting osd_memory_target on smithi131 to 5354M 2024-02-05T07:47:31.181 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.181 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:31.181 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:31.181 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.181 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:47:31.181 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:31.181 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:31.181 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.181 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:31.181 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.181 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: Detected new or changed devices on smithi131 2024-02-05T07:47:31.181 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.181 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:31.181 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: osdmap e21: 3 total, 3 up, 3 in 2024-02-05T07:47:31.181 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.182 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:31.182 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:31.182 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:31.182 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: Adjusting osd_memory_target on smithi131 to 5354M 2024-02-05T07:47:31.182 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.182 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:31.182 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:31.182 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.182 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:47:31.182 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:31.182 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:31.182 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.182 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:31.182 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.432 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 07:47:31 smithi131 sudo[62997]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-05T07:47:31.432 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 07:47:31 smithi131 sudo[62997]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:47:31.432 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 07:47:31 smithi131 sudo[62997]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:47:31.432 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 07:47:31 smithi131 sudo[62997]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:47:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: Detected new or changed devices on smithi131 2024-02-05T07:47:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: osdmap e21: 3 total, 3 up, 3 in 2024-02-05T07:47:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:31.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: Adjusting osd_memory_target on smithi131 to 5354M 2024-02-05T07:47:31.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:31.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:31.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:47:31.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:31.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:31.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:31.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:31.696 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 07:47:31 smithi131 sudo[63020]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-05T07:47:31.696 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 07:47:31 smithi131 sudo[63020]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:47:31.696 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 07:47:31 smithi131 sudo[63020]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:47:31.696 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 07:47:31 smithi131 sudo[63020]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:47:31.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.858+0000 7f1119115700 1 -- 172.21.15.131:0/2357691885 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1114101e40 msgr2=0x7f11141022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:47:31.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.858+0000 7f1119115700 1 --2- 172.21.15.131:0/2357691885 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1114101e40 0x7f11141022a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1108009990 tx=0x7f1108009ca0 comp rx=0 tx=0).stop 2024-02-05T07:47:31.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.858+0000 7f1119115700 1 -- 172.21.15.131:0/2357691885 shutdown_connections 2024-02-05T07:47:31.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.858+0000 7f1119115700 1 --2- 172.21.15.131:0/2357691885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1114100c40 0x7f1114101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:31.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.858+0000 7f1119115700 1 --2- 172.21.15.131:0/2357691885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11141027e0 0x7f1114109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:31.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.858+0000 7f1119115700 1 --2- 172.21.15.131:0/2357691885 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1114101e40 0x7f11141022a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:31.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.858+0000 7f1119115700 1 -- 172.21.15.131:0/2357691885 >> 172.21.15.131:0/2357691885 conn(0x7f11140fc1e0 msgr2=0x7f11140fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:47:31.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.859+0000 7f1119115700 1 -- 172.21.15.131:0/2357691885 shutdown_connections 2024-02-05T07:47:31.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.859+0000 7f1119115700 1 -- 172.21.15.131:0/2357691885 wait complete. 2024-02-05T07:47:31.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.859+0000 7f1119115700 1 Processor -- start 2024-02-05T07:47:31.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.859+0000 7f1119115700 1 -- start start 2024-02-05T07:47:31.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f1119115700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1114100c40 0x7f1114197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:47:31.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f1119115700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1114101e40 0x7f1114197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:47:31.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f1119115700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11141027e0 0x7f111419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:47:31.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f1119115700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111419eec0 con 0x7f11141027e0 2024-02-05T07:47:31.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f1119115700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111419f000 con 0x7f1114101e40 2024-02-05T07:47:31.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f1119115700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111419f190 con 0x7f1114100c40 2024-02-05T07:47:31.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f1112d9d700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1114100c40 0x7f1114197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:47:31.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f111359e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11141027e0 0x7f111419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:47:31.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f111259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1114101e40 0x7f1114197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:47:31.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f1112d9d700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1114100c40 0x7f1114197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:47808/0 (socket says 172.21.15.131:47808) 2024-02-05T07:47:31.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f111259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1114101e40 0x7f1114197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:36150/0 (socket says 172.21.15.131:36150) 2024-02-05T07:47:31.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f1112d9d700 1 -- 172.21.15.131:0/2389726198 learned_addr learned my addr 172.21.15.131:0/2389726198 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:47:31.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f111259c700 1 -- 172.21.15.131:0/2389726198 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1114100c40 msgr2=0x7f1114197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:47:31.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f111259c700 1 --2- 172.21.15.131:0/2389726198 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1114100c40 0x7f1114197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:31.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f111259c700 1 -- 172.21.15.131:0/2389726198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11141027e0 msgr2=0x7f111419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:47:31.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f111259c700 1 --2- 172.21.15.131:0/2389726198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11141027e0 0x7f111419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:31.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f111259c700 1 -- 172.21.15.131:0/2389726198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11080095f0 con 0x7f1114101e40 2024-02-05T07:47:31.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.860+0000 7f111259c700 1 --2- 172.21.15.131:0/2389726198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1114101e40 0x7f1114197870 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1108000c00 tx=0x7f1108004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:47:31.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.861+0000 7f10fbfff700 1 -- 172.21.15.131:0/2389726198 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110801d070 con 0x7f1114101e40 2024-02-05T07:47:31.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.861+0000 7f1119115700 1 -- 172.21.15.131:0/2389726198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111419f410 con 0x7f1114101e40 2024-02-05T07:47:31.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.861+0000 7f1119115700 1 -- 172.21.15.131:0/2389726198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111419f880 con 0x7f1114101e40 2024-02-05T07:47:31.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.861+0000 7f10fbfff700 1 -- 172.21.15.131:0/2389726198 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f1108004090 con 0x7f1114101e40 2024-02-05T07:47:31.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.861+0000 7f10fbfff700 1 -- 172.21.15.131:0/2389726198 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108022ce0 con 0x7f1114101e40 2024-02-05T07:47:31.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.862+0000 7f1119115700 1 -- 172.21.15.131:0/2389726198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1114108a20 con 0x7f1114101e40 2024-02-05T07:47:31.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.862+0000 7f10fbfff700 1 -- 172.21.15.131:0/2389726198 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 14) v1 ==== 91316+0+0 (secure 0 0 0) 0x7f1108003be0 con 0x7f1114101e40 2024-02-05T07:47:31.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.863+0000 7f10fbfff700 1 --2- 172.21.15.131:0/2389726198 >> v2:172.21.15.131:6800/760045032 conn(0x7f10fc06d900 0x7f10fc06fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:47:31.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.863+0000 7f10fbfff700 1 -- 172.21.15.131:0/2389726198 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 2511+0+0 (secure 0 0 0) 0x7f110808d520 con 0x7f1114101e40 2024-02-05T07:47:31.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.863+0000 7f1112d9d700 1 --2- 172.21.15.131:0/2389726198 >> v2:172.21.15.131:6800/760045032 conn(0x7f10fc06d900 0x7f10fc06fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:47:31.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.863+0000 7f1112d9d700 1 --2- 172.21.15.131:0/2389726198 >> v2:172.21.15.131:6800/760045032 conn(0x7f10fc06d900 0x7f10fc06fdc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f110400ace0 tx=0x7f110400a430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:47:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:31.868+0000 7f10fbfff700 1 -- 172.21.15.131:0/2389726198 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1108057f40 con 0x7f1114101e40 2024-02-05T07:47:31.947 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 07:47:31 smithi131 sudo[63042]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-05T07:47:31.947 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 07:47:31 smithi131 sudo[63042]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:47:31.947 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 07:47:31 smithi131 sudo[63042]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:47:31.947 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 07:47:31 smithi131 sudo[63042]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:47:32.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:32.079+0000 7f1119115700 1 -- 172.21.15.131:0/2389726198 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1114062160 con 0x7f10fc06d900 2024-02-05T07:47:32.206 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 sudo[63072]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-05T07:47:32.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 sudo[63072]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:47:32.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 sudo[63072]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:47:32.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:31 smithi131 sudo[63072]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:47:32.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:32.221+0000 7f10fbfff700 1 -- 172.21.15.131:0/2389726198 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f11080579f0 con 0x7f1114101e40 2024-02-05T07:47:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:32 smithi157 ceph-mon[33345]: osdmap e22: 3 total, 3 up, 3 in 2024-02-05T07:47:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:32 smithi157 ceph-mon[33345]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:47:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:32 smithi157 sudo[36177]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-05T07:47:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:32 smithi157 sudo[36177]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:47:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:32 smithi157 sudo[36177]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:47:32.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:32 smithi131 ceph-mon[41654]: osdmap e22: 3 total, 3 up, 3 in 2024-02-05T07:47:32.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:32 smithi131 ceph-mon[41654]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:47:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:32 smithi131 sudo[63076]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-05T07:47:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:32 smithi131 sudo[63076]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:47:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:32 smithi131 sudo[63076]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:47:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:32 smithi131 sudo[63076]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:47:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:32 smithi131 ceph-mon[29507]: osdmap e22: 3 total, 3 up, 3 in 2024-02-05T07:47:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:32 smithi131 ceph-mon[29507]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:47:32.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:32 smithi157 sudo[36177]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:47:32.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:32 smithi157 sudo[36180]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-05T07:47:32.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:32 smithi157 sudo[36180]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:47:32.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:32 smithi157 sudo[36180]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:47:32.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:32 smithi157 sudo[36180]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:47:33.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:32 smithi131 sudo[63221]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-05T07:47:33.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:32 smithi131 sudo[63221]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:47:33.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:32 smithi131 sudo[63221]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:47:33.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 sudo[63221]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:47:33.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:47:33.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:47:33.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:47:33.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:47:33.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:47:33.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:47:33.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:47:33.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-05T07:47:33.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:47:33.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:47:33.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:47:33.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:47:33.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:33.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:33.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:47:33.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:33.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:47:33.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:47:33.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:47:33.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 sudo[63243]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-05T07:47:33.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 sudo[63243]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:47:33.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 sudo[63243]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:47:33.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 sudo[63243]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:47:33.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:47:33.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:47:33.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:47:33.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:47:33.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:47:33.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:47:33.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:47:33.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-05T07:47:33.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:47:33.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:47:33.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:47:33.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:47:33.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:33.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:33.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:47:33.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:33.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:47:33.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:47:33.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:47:33.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:47:33.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:47:33.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:47:33.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:47:33.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:47:33.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:47:33.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:47:33.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-05T07:47:33.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:47:33.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:47:33.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:47:33.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:47:33.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:33.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:33.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:47:33.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:33.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:47:33.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:47:33.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:47:34.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:34 smithi131 ceph-mon[41654]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:47:34.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:34 smithi131 ceph-mon[41654]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:47:34.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:34 smithi131 ceph-mon[29507]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:47:34.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:34 smithi131 ceph-mon[29507]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:47:34.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:34 smithi157 ceph-mon[33345]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:47:34.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:34 smithi157 ceph-mon[33345]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:47:35.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:35 smithi131 ceph-mon[29507]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:35.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3700854166' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e262dfc-6d5a-4f3f-b605-6c92e6839f62"}]: dispatch 2024-02-05T07:47:35.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3700854166' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e262dfc-6d5a-4f3f-b605-6c92e6839f62"}]': finished 2024-02-05T07:47:35.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:35 smithi131 ceph-mon[29507]: osdmap e23: 4 total, 3 up, 4 in 2024-02-05T07:47:35.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:35.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:35 smithi131 ceph-mon[41654]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:35.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3700854166' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e262dfc-6d5a-4f3f-b605-6c92e6839f62"}]: dispatch 2024-02-05T07:47:35.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3700854166' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e262dfc-6d5a-4f3f-b605-6c92e6839f62"}]': finished 2024-02-05T07:47:35.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:35 smithi131 ceph-mon[41654]: osdmap e23: 4 total, 3 up, 4 in 2024-02-05T07:47:35.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:35 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:35.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:35 smithi157 ceph-mon[33345]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:35.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3700854166' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e262dfc-6d5a-4f3f-b605-6c92e6839f62"}]: dispatch 2024-02-05T07:47:35.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3700854166' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e262dfc-6d5a-4f3f-b605-6c92e6839f62"}]': finished 2024-02-05T07:47:35.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:35 smithi157 ceph-mon[33345]: osdmap e23: 4 total, 3 up, 4 in 2024-02-05T07:47:35.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:35 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:36.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/148680757' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:47:36.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/148680757' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:47:36.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/148680757' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:47:37.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:37 smithi131 ceph-mon[29507]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:37.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:37 smithi131 ceph-mon[41654]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:37.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:37 smithi157 ceph-mon[33345]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:39.932 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:39 smithi131 ceph-mon[41654]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-05T07:47:39.933 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:39 smithi131 ceph-mon[29507]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-05T07:47:39.970 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:39 smithi157 ceph-mon[33345]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-05T07:47:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:41 smithi157 ceph-mon[33345]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-05T07:47:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:41 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-05T07:47:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:41 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:42.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:41 smithi131 ceph-mon[29507]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-05T07:47:42.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:41 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-05T07:47:42.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:41 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:42.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:41 smithi131 ceph-mon[41654]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-05T07:47:42.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:41 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-05T07:47:42.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:41 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:42 smithi157 ceph-mon[33345]: Deploying daemon osd.3 on smithi131 2024-02-05T07:47:43.011 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:42 smithi131 ceph-mon[41654]: Deploying daemon osd.3 on smithi131 2024-02-05T07:47:43.011 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:42 smithi131 ceph-mon[29507]: Deploying daemon osd.3 on smithi131 2024-02-05T07:47:43.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:43 smithi131 ceph-mon[29507]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-05T07:47:43.962 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:43 smithi131 ceph-mon[41654]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-05T07:47:43.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:43 smithi157 ceph-mon[33345]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-05T07:47:45.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:45 smithi131 ceph-mon[29507]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-05T07:47:45.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:45 smithi131 ceph-mon[41654]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-05T07:47:45.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:45 smithi157 ceph-mon[33345]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-05T07:47:47.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:47 smithi131 ceph-mon[41654]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 67 KiB/s wr, 4 op/s 2024-02-05T07:47:47.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:47 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:47.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:47 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:47.890 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:47 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:47:47.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:47 smithi131 ceph-mon[29507]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 67 KiB/s wr, 4 op/s 2024-02-05T07:47:47.890 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:47 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:47.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:47 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:47.891 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:47 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:47:47.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:47 smithi157 ceph-mon[33345]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 67 KiB/s wr, 4 op/s 2024-02-05T07:47:47.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:47 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:47.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:47 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:47.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:47 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:47:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:49 smithi157 ceph-mon[33345]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-05T07:47:49.976 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:49 smithi131 ceph-mon[29507]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-05T07:47:49.976 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:49 smithi131 ceph-mon[41654]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-05T07:47:50.667 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 3 on host 'smithi131' 2024-02-05T07:47:50.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:50.665+0000 7f10fbfff700 1 -- 172.21.15.131:0/2389726198 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1114062160 con 0x7f10fc06d900 2024-02-05T07:47:50.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:50.667+0000 7f1119115700 1 -- 172.21.15.131:0/2389726198 >> v2:172.21.15.131:6800/760045032 conn(0x7f10fc06d900 msgr2=0x7f10fc06fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:47:50.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:50.667+0000 7f1119115700 1 --2- 172.21.15.131:0/2389726198 >> v2:172.21.15.131:6800/760045032 conn(0x7f10fc06d900 0x7f10fc06fdc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f110400ace0 tx=0x7f110400a430 comp rx=0 tx=0).stop 2024-02-05T07:47:50.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:50.667+0000 7f1119115700 1 -- 172.21.15.131:0/2389726198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1114101e40 msgr2=0x7f1114197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:47:50.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:50.667+0000 7f1119115700 1 --2- 172.21.15.131:0/2389726198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1114101e40 0x7f1114197870 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1108000c00 tx=0x7f1108004730 comp rx=0 tx=0).stop 2024-02-05T07:47:50.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:50.667+0000 7f1119115700 1 -- 172.21.15.131:0/2389726198 shutdown_connections 2024-02-05T07:47:50.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:50.667+0000 7f1119115700 1 --2- 172.21.15.131:0/2389726198 >> v2:172.21.15.131:6800/760045032 conn(0x7f10fc06d900 0x7f10fc06fdc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:50.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:50.667+0000 7f1119115700 1 --2- 172.21.15.131:0/2389726198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11141027e0 0x7f111419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:50.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:50.667+0000 7f1119115700 1 --2- 172.21.15.131:0/2389726198 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1114100c40 0x7f1114197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:50.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:50.667+0000 7f1119115700 1 --2- 172.21.15.131:0/2389726198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1114101e40 0x7f1114197870 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:50.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:50.668+0000 7f1119115700 1 -- 172.21.15.131:0/2389726198 >> 172.21.15.131:0/2389726198 conn(0x7f11140fc1e0 msgr2=0x7f11140fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:47:50.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:50.668+0000 7f1119115700 1 -- 172.21.15.131:0/2389726198 shutdown_connections 2024-02-05T07:47:50.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:47:50.668+0000 7f1119115700 1 -- 172.21.15.131:0/2389726198 wait complete. 2024-02-05T07:47:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:50 smithi157 ceph-mon[33345]: from='osd.3 v2:172.21.15.131:6813/3290626285' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-05T07:47:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:50 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:50 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:51.009 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:50 smithi131 ceph-mon[41654]: from='osd.3 v2:172.21.15.131:6813/3290626285' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-05T07:47:51.009 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:50 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:51.009 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:50 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:51.009 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:50 smithi131 ceph-mon[29507]: from='osd.3 v2:172.21.15.131:6813/3290626285' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-05T07:47:51.009 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:50 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:51.009 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:50 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:51.260 DEBUG:teuthology.orchestra.run.smithi131:osd.3> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.3.service 2024-02-05T07:47:51.263 INFO:tasks.cephadm:Deploying osd.4 on smithi157 with /dev/vg_nvme/lv_4... 2024-02-05T07:47:51.263 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-05T07:47:51.507 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:47:52.166 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:51 smithi131 ceph-mon[41654]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:52.166 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:51 smithi131 ceph-mon[41654]: from='osd.3 v2:172.21.15.131:6813/3290626285' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-05T07:47:52.166 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:51 smithi131 ceph-mon[41654]: osdmap e24: 4 total, 3 up, 4 in 2024-02-05T07:47:52.166 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:51 smithi131 ceph-mon[41654]: from='osd.3 v2:172.21.15.131:6813/3290626285' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-02-05T07:47:52.166 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:51 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:52.166 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:51 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:52.166 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:51 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:52.167 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:51 smithi131 ceph-mon[29507]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:52.167 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:51 smithi131 ceph-mon[29507]: from='osd.3 v2:172.21.15.131:6813/3290626285' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-05T07:47:52.167 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:51 smithi131 ceph-mon[29507]: osdmap e24: 4 total, 3 up, 4 in 2024-02-05T07:47:52.167 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:51 smithi131 ceph-mon[29507]: from='osd.3 v2:172.21.15.131:6813/3290626285' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-02-05T07:47:52.167 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:51 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:52.167 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:51 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:52.167 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:51 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:52.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:51 smithi157 ceph-mon[33345]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:52.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:51 smithi157 ceph-mon[33345]: from='osd.3 v2:172.21.15.131:6813/3290626285' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-05T07:47:52.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:51 smithi157 ceph-mon[33345]: osdmap e24: 4 total, 3 up, 4 in 2024-02-05T07:47:52.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:51 smithi157 ceph-mon[33345]: from='osd.3 v2:172.21.15.131:6813/3290626285' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-02-05T07:47:52.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:51 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:52.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:51 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:52.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:51 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:52.439 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 07:47:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3[65147]: 2024-02-05T07:47:52.171+0000 7f18d45c8700 -1 osd.3 0 waiting for initial osdmap 2024-02-05T07:47:52.439 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 07:47:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3[65147]: 2024-02-05T07:47:52.175+0000 7f18cc3ba700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T07:47:53.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:52 smithi131 ceph-mon[29507]: from='osd.3 v2:172.21.15.131:6813/3290626285' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-02-05T07:47:53.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:52 smithi131 ceph-mon[29507]: osdmap e25: 4 total, 3 up, 4 in 2024-02-05T07:47:53.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:52 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:53.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:52 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:53.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:52 smithi131 ceph-mon[29507]: from='osd.3 v2:172.21.15.131:6813/3290626285' entity='osd.3' 2024-02-05T07:47:53.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:52 smithi131 ceph-mon[29507]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:53.083 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:52 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:53.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:52 smithi131 ceph-mon[41654]: from='osd.3 v2:172.21.15.131:6813/3290626285' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-02-05T07:47:53.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:52 smithi131 ceph-mon[41654]: osdmap e25: 4 total, 3 up, 4 in 2024-02-05T07:47:53.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:52 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:53.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:52 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:53.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:52 smithi131 ceph-mon[41654]: from='osd.3 v2:172.21.15.131:6813/3290626285' entity='osd.3' 2024-02-05T07:47:53.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:52 smithi131 ceph-mon[41654]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:53.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:52 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:52 smithi157 ceph-mon[33345]: from='osd.3 v2:172.21.15.131:6813/3290626285' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-02-05T07:47:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:52 smithi157 ceph-mon[33345]: osdmap e25: 4 total, 3 up, 4 in 2024-02-05T07:47:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:52 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:52 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:52 smithi157 ceph-mon[33345]: from='osd.3 v2:172.21.15.131:6813/3290626285' entity='osd.3' 2024-02-05T07:47:53.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:52 smithi157 ceph-mon[33345]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:47:53.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:52 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-05T07:47:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-05T07:47:53.256 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-02-05T07:47:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-02-05T07:47:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00820462 s, 1.3 GB/s 2024-02-05T07:47:53.256 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 2024-02-05T07:47:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-02-05T07:47:53.925 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi157:vg_nvme/lv_4 2024-02-05T07:47:54.165 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:47:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:54 smithi157 ceph-mon[33345]: purged_snaps scrub starts 2024-02-05T07:47:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:54 smithi157 ceph-mon[33345]: purged_snaps scrub ok 2024-02-05T07:47:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:54 smithi157 ceph-mon[33345]: osd.3 v2:172.21.15.131:6813/3290626285 boot 2024-02-05T07:47:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:54 smithi157 ceph-mon[33345]: osdmap e26: 4 total, 4 up, 4 in 2024-02-05T07:47:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:54 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:54.515 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:54 smithi131 ceph-mon[29507]: purged_snaps scrub starts 2024-02-05T07:47:54.515 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:54 smithi131 ceph-mon[29507]: purged_snaps scrub ok 2024-02-05T07:47:54.515 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:54 smithi131 ceph-mon[29507]: osd.3 v2:172.21.15.131:6813/3290626285 boot 2024-02-05T07:47:54.515 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:54 smithi131 ceph-mon[29507]: osdmap e26: 4 total, 4 up, 4 in 2024-02-05T07:47:54.515 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:54 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:54.515 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:54 smithi131 ceph-mon[41654]: purged_snaps scrub starts 2024-02-05T07:47:54.515 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:54 smithi131 ceph-mon[41654]: purged_snaps scrub ok 2024-02-05T07:47:54.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:54 smithi131 ceph-mon[41654]: osd.3 v2:172.21.15.131:6813/3290626285 boot 2024-02-05T07:47:54.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:54 smithi131 ceph-mon[41654]: osdmap e26: 4 total, 4 up, 4 in 2024-02-05T07:47:54.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:54 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:47:55.284 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:55 smithi157 ceph-mon[33345]: osdmap e27: 4 total, 4 up, 4 in 2024-02-05T07:47:55.284 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:55 smithi157 ceph-mon[33345]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:47:55.284 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:55.284 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:55.284 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:55.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:55.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:55.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:55.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:55.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:55.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:55.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:55.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:47:55.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:55.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:55.285 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:55.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[29507]: osdmap e27: 4 total, 4 up, 4 in 2024-02-05T07:47:55.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[29507]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:47:55.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:55.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:55.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:55.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:55.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:55.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:55.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:55.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:55.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:55.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:55.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:47:55.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:55.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:55.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:55.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[41654]: osdmap e27: 4 total, 4 up, 4 in 2024-02-05T07:47:55.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[41654]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:47:55.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:55.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:55.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:55.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:55.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:55.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:47:55.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:55.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:55.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:55.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:55.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:47:55.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:55.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:55.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:47:55.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.750+0000 7f81fc14e700 1 -- 172.21.15.157:0/1489006488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81f4101e40 msgr2=0x7f81f41022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:47:55.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.750+0000 7f81fc14e700 1 --2- 172.21.15.157:0/1489006488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81f4101e40 0x7f81f41022a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f81e80099c0 tx=0x7f81e8009cd0 comp rx=0 tx=0).stop 2024-02-05T07:47:55.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.750+0000 7f81fc14e700 1 -- 172.21.15.157:0/1489006488 shutdown_connections 2024-02-05T07:47:55.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.750+0000 7f81fc14e700 1 --2- 172.21.15.157:0/1489006488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f4100c40 0x7f81f4101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:55.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.750+0000 7f81fc14e700 1 --2- 172.21.15.157:0/1489006488 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f81f41027e0 0x7f81f4109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:55.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.750+0000 7f81fc14e700 1 --2- 172.21.15.157:0/1489006488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81f4101e40 0x7f81f41022a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:55.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.750+0000 7f81fc14e700 1 -- 172.21.15.157:0/1489006488 >> 172.21.15.157:0/1489006488 conn(0x7f81f40fc1e0 msgr2=0x7f81f40fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:47:55.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.750+0000 7f81fc14e700 1 -- 172.21.15.157:0/1489006488 shutdown_connections 2024-02-05T07:47:55.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.750+0000 7f81fc14e700 1 -- 172.21.15.157:0/1489006488 wait complete. 2024-02-05T07:47:55.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.750+0000 7f81fc14e700 1 Processor -- start 2024-02-05T07:47:55.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81fc14e700 1 -- start start 2024-02-05T07:47:55.753 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81fc14e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81f4100c40 0x7f81f41996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:47:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81fc14e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f4101e40 0x7f81f4199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:47:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81fc14e700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f81f41027e0 0x7f81f41a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:47:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81fc14e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f41a1250 con 0x7f81f4101e40 2024-02-05T07:47:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81fc14e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f41a1390 con 0x7f81f4100c40 2024-02-05T07:47:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81fc14e700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f41a1520 con 0x7f81f41027e0 2024-02-05T07:47:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81f9eea700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81f4100c40 0x7f81f41996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:47:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81f9eea700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81f4100c40 0x7f81f41996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42954/0 (socket says 172.21.15.157:42954) 2024-02-05T07:47:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81f9eea700 1 -- 172.21.15.157:0/3397540209 learned_addr learned my addr 172.21.15.157:0/3397540209 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:47:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81f96e9700 1 --2- 172.21.15.157:0/3397540209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f4101e40 0x7f81f4199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:47:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81fa6eb700 1 --2- 172.21.15.157:0/3397540209 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f81f41027e0 0x7f81f41a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:47:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81f9eea700 1 -- 172.21.15.157:0/3397540209 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f81f41027e0 msgr2=0x7f81f41a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:47:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81f9eea700 1 --2- 172.21.15.157:0/3397540209 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f81f41027e0 0x7f81f41a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81f9eea700 1 -- 172.21.15.157:0/3397540209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f4101e40 msgr2=0x7f81f4199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:47:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81f9eea700 1 --2- 172.21.15.157:0/3397540209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f4101e40 0x7f81f4199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:47:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81f9eea700 1 -- 172.21.15.157:0/3397540209 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e80095f0 con 0x7f81f4100c40 2024-02-05T07:47:55.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81f9eea700 1 --2- 172.21.15.157:0/3397540209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81f4100c40 0x7f81f41996c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f81e400b9c0 tx=0x7f81e400bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:47:55.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81f2ffd700 1 -- 172.21.15.157:0/3397540209 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e400d8e0 con 0x7f81f4100c40 2024-02-05T07:47:55.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.751+0000 7f81f2ffd700 1 -- 172.21.15.157:0/3397540209 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f81e4009900 con 0x7f81f4100c40 2024-02-05T07:47:55.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.752+0000 7f81fc14e700 1 -- 172.21.15.157:0/3397540209 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f41a1800 con 0x7f81f4100c40 2024-02-05T07:47:55.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.752+0000 7f81f2ffd700 1 -- 172.21.15.157:0/3397540209 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e400c400 con 0x7f81f4100c40 2024-02-05T07:47:55.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.752+0000 7f81fc14e700 1 -- 172.21.15.157:0/3397540209 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f41a1dd0 con 0x7f81f4100c40 2024-02-05T07:47:55.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.752+0000 7f81fc14e700 1 -- 172.21.15.157:0/3397540209 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f4106810 con 0x7f81f4100c40 2024-02-05T07:47:55.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.753+0000 7f81f2ffd700 1 -- 172.21.15.157:0/3397540209 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f81e400c560 con 0x7f81f4100c40 2024-02-05T07:47:55.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.753+0000 7f81f2ffd700 1 --2- 172.21.15.157:0/3397540209 >> v2:172.21.15.131:6800/760045032 conn(0x7f81dc06d990 0x7f81dc06fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:47:55.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.753+0000 7f81f2ffd700 1 -- 172.21.15.157:0/3397540209 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 2803+0+0 (secure 0 0 0) 0x7f81e4011070 con 0x7f81f4100c40 2024-02-05T07:47:55.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.754+0000 7f81f96e9700 1 --2- 172.21.15.157:0/3397540209 >> v2:172.21.15.131:6800/760045032 conn(0x7f81dc06d990 0x7f81dc06fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:47:55.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.754+0000 7f81f96e9700 1 --2- 172.21.15.157:0/3397540209 >> v2:172.21.15.131:6800/760045032 conn(0x7f81dc06d990 0x7f81dc06fe50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009990 tx=0x7f81e8005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:47:55.759 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.758+0000 7f81f2ffd700 1 -- 172.21.15.157:0/3397540209 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f81e404fd20 con 0x7f81f4100c40 2024-02-05T07:47:55.960 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:47:55.959+0000 7f81fc14e700 1 -- 172.21.15.157:0/3397540209 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f81f410bf50 con 0x7f81dc06d990 2024-02-05T07:47:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:56 smithi157 ceph-mon[33345]: Detected new or changed devices on smithi131 2024-02-05T07:47:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:56 smithi157 ceph-mon[33345]: Adjusting osd_memory_target on smithi131 to 4016M 2024-02-05T07:47:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:56 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:47:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:56 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:47:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:56 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:56 smithi131 ceph-mon[29507]: Detected new or changed devices on smithi131 2024-02-05T07:47:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:56 smithi131 ceph-mon[29507]: Adjusting osd_memory_target on smithi131 to 4016M 2024-02-05T07:47:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:56 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:47:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:56 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:47:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:56 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:56.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:56 smithi131 ceph-mon[41654]: Detected new or changed devices on smithi131 2024-02-05T07:47:56.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:56 smithi131 ceph-mon[41654]: Adjusting osd_memory_target on smithi131 to 4016M 2024-02-05T07:47:56.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:47:56.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:47:56.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:57.434 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:57 smithi157 ceph-mon[33345]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:47:57.434 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:57 smithi157 ceph-mon[33345]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:47:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:57 smithi131 ceph-mon[29507]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:47:57.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:57 smithi131 ceph-mon[29507]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:47:57.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:57 smithi131 ceph-mon[41654]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:47:57.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:57 smithi131 ceph-mon[41654]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:47:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.157:0/2884849269' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6925a13-9a72-42bd-97b2-970f9e709dad"}]: dispatch 2024-02-05T07:47:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6925a13-9a72-42bd-97b2-970f9e709dad"}]: dispatch 2024-02-05T07:47:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6925a13-9a72-42bd-97b2-970f9e709dad"}]': finished 2024-02-05T07:47:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:58 smithi157 ceph-mon[33345]: osdmap e28: 5 total, 4 up, 5 in 2024-02-05T07:47:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:58 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:47:59.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.157:0/2884849269' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6925a13-9a72-42bd-97b2-970f9e709dad"}]: dispatch 2024-02-05T07:47:59.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6925a13-9a72-42bd-97b2-970f9e709dad"}]: dispatch 2024-02-05T07:47:59.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6925a13-9a72-42bd-97b2-970f9e709dad"}]': finished 2024-02-05T07:47:59.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:58 smithi131 ceph-mon[29507]: osdmap e28: 5 total, 4 up, 5 in 2024-02-05T07:47:59.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:58 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:47:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.157:0/2884849269' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6925a13-9a72-42bd-97b2-970f9e709dad"}]: dispatch 2024-02-05T07:47:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6925a13-9a72-42bd-97b2-970f9e709dad"}]: dispatch 2024-02-05T07:47:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6925a13-9a72-42bd-97b2-970f9e709dad"}]': finished 2024-02-05T07:47:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:58 smithi131 ceph-mon[41654]: osdmap e28: 5 total, 4 up, 5 in 2024-02-05T07:47:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:58 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:47:59.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:59 smithi157 ceph-mon[33345]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:47:59.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:47:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.157:0/187927111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:48:00.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:59 smithi131 ceph-mon[29507]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:00.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:47:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.157:0/187927111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:48:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:59 smithi131 ceph-mon[41654]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:47:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.157:0/187927111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:48:01.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:01 smithi157 ceph-mon[33345]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:02.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:01 smithi131 ceph-mon[29507]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:01 smithi131 ceph-mon[41654]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:03.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:03 smithi157 ceph-mon[33345]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:04.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:03 smithi131 ceph-mon[29507]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:04.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:03 smithi131 ceph-mon[41654]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:05.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:05 smithi157 ceph-mon[33345]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:05.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-05T07:48:05.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:06.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:05 smithi131 ceph-mon[29507]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:06.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-05T07:48:06.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:05 smithi131 ceph-mon[41654]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-05T07:48:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:06.854 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:06 smithi157 ceph-mon[33345]: Deploying daemon osd.4 on smithi157 2024-02-05T07:48:07.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:06 smithi131 ceph-mon[29507]: Deploying daemon osd.4 on smithi157 2024-02-05T07:48:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:06 smithi131 ceph-mon[41654]: Deploying daemon osd.4 on smithi157 2024-02-05T07:48:07.851 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:07 smithi157 ceph-mon[33345]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:08.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:07 smithi131 ceph-mon[29507]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:08.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:07 smithi131 ceph-mon[41654]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:09.877 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:09 smithi157 ceph-mon[33345]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:09 smithi131 ceph-mon[29507]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:09 smithi131 ceph-mon[41654]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:11.718 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:11 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:11.718 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:11 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:48:11.719 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:11 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:11.719 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:11 smithi157 ceph-mon[33345]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:11 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:11 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:48:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:11 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:11 smithi131 ceph-mon[41654]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:11.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:11 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:11.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:11 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:48:11.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:11 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:11.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:11 smithi131 ceph-mon[29507]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:13.671 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:13.669+0000 7f81f2ffd700 1 -- 172.21.15.157:0/3397540209 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f81f410bf50 con 0x7f81dc06d990 2024-02-05T07:48:13.672 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 4 on host 'smithi157' 2024-02-05T07:48:13.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:13.671+0000 7f81fc14e700 1 -- 172.21.15.157:0/3397540209 >> v2:172.21.15.131:6800/760045032 conn(0x7f81dc06d990 msgr2=0x7f81dc06fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:48:13.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:13.671+0000 7f81fc14e700 1 --2- 172.21.15.157:0/3397540209 >> v2:172.21.15.131:6800/760045032 conn(0x7f81dc06d990 0x7f81dc06fe50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009990 tx=0x7f81e8005ee0 comp rx=0 tx=0).stop 2024-02-05T07:48:13.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:13.671+0000 7f81fc14e700 1 -- 172.21.15.157:0/3397540209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81f4100c40 msgr2=0x7f81f41996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:48:13.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:13.671+0000 7f81fc14e700 1 --2- 172.21.15.157:0/3397540209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81f4100c40 0x7f81f41996c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f81e400b9c0 tx=0x7f81e400bcd0 comp rx=0 tx=0).stop 2024-02-05T07:48:13.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:13.671+0000 7f81fc14e700 1 -- 172.21.15.157:0/3397540209 shutdown_connections 2024-02-05T07:48:13.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:13.671+0000 7f81fc14e700 1 --2- 172.21.15.157:0/3397540209 >> v2:172.21.15.131:6800/760045032 conn(0x7f81dc06d990 0x7f81dc06fe50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:13.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:13.671+0000 7f81fc14e700 1 --2- 172.21.15.157:0/3397540209 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f81f41027e0 0x7f81f41a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:13.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:13.671+0000 7f81fc14e700 1 --2- 172.21.15.157:0/3397540209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81f4100c40 0x7f81f41996c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:13.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:13.671+0000 7f81fc14e700 1 --2- 172.21.15.157:0/3397540209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81f4101e40 0x7f81f4199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:13.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:13.671+0000 7f81fc14e700 1 -- 172.21.15.157:0/3397540209 >> 172.21.15.157:0/3397540209 conn(0x7f81f40fc1e0 msgr2=0x7f81f40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:48:13.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:13.671+0000 7f81fc14e700 1 -- 172.21.15.157:0/3397540209 shutdown_connections 2024-02-05T07:48:13.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:13.671+0000 7f81fc14e700 1 -- 172.21.15.157:0/3397540209 wait complete. 2024-02-05T07:48:13.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:13 smithi157 ceph-mon[33345]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:13.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:13 smithi157 ceph-mon[33345]: from='osd.4 v2:172.21.15.157:6800/1828659015' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-05T07:48:13.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:13 smithi157 ceph-mon[33345]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-05T07:48:13.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:13 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:13.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:13 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:14.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:13 smithi131 ceph-mon[29507]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:14.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:13 smithi131 ceph-mon[29507]: from='osd.4 v2:172.21.15.157:6800/1828659015' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-05T07:48:14.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:13 smithi131 ceph-mon[29507]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-05T07:48:14.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:13 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:14.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:13 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:13 smithi131 ceph-mon[41654]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:13 smithi131 ceph-mon[41654]: from='osd.4 v2:172.21.15.157:6800/1828659015' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-05T07:48:14.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:13 smithi131 ceph-mon[41654]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-05T07:48:14.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:13 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:14.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:13 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:14.290 DEBUG:teuthology.orchestra.run.smithi157:osd.4> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.4.service 2024-02-05T07:48:14.293 INFO:tasks.cephadm:Deploying osd.5 on smithi157 with /dev/vg_nvme/lv_3... 2024-02-05T07:48:14.293 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-05T07:48:14.861 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:14 smithi157 ceph-mon[33345]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-05T07:48:14.861 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:14 smithi157 ceph-mon[33345]: osdmap e29: 5 total, 4 up, 5 in 2024-02-05T07:48:14.861 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:14 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:48:14.861 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:14 smithi157 ceph-mon[33345]: from='osd.4 v2:172.21.15.157:6800/1828659015' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:48:14.862 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:14 smithi157 ceph-mon[33345]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:48:14.862 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:14 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:14.862 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:14 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:14.862 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 07:48:14 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T07:48:14.707+0000 7fbb50d6f700 -1 osd.4 0 waiting for initial osdmap 2024-02-05T07:48:14.862 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 07:48:14 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T07:48:14.713+0000 7fbb4cb69700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T07:48:15.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:14 smithi131 ceph-mon[29507]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-05T07:48:15.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:14 smithi131 ceph-mon[29507]: osdmap e29: 5 total, 4 up, 5 in 2024-02-05T07:48:15.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:14 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:48:15.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:14 smithi131 ceph-mon[29507]: from='osd.4 v2:172.21.15.157:6800/1828659015' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:48:15.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:14 smithi131 ceph-mon[29507]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:48:15.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:14 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:15.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:14 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:14 smithi131 ceph-mon[41654]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-05T07:48:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:14 smithi131 ceph-mon[41654]: osdmap e29: 5 total, 4 up, 5 in 2024-02-05T07:48:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:14 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:48:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:14 smithi131 ceph-mon[41654]: from='osd.4 v2:172.21.15.157:6800/1828659015' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:48:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:14 smithi131 ceph-mon[41654]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:48:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:14 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:14 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:15.076 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:48:15.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:15 smithi157 ceph-mon[33345]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:15.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:15 smithi157 ceph-mon[33345]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-02-05T07:48:15.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:15 smithi157 ceph-mon[33345]: osdmap e30: 5 total, 4 up, 5 in 2024-02-05T07:48:15.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:15 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:48:15.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:15 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:48:16.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:15 smithi131 ceph-mon[29507]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:16.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:15 smithi131 ceph-mon[29507]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-02-05T07:48:16.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:15 smithi131 ceph-mon[29507]: osdmap e30: 5 total, 4 up, 5 in 2024-02-05T07:48:16.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:15 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:48:16.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:15 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:48:16.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:15 smithi131 ceph-mon[41654]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-05T07:48:16.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:15 smithi131 ceph-mon[41654]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-02-05T07:48:16.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:15 smithi131 ceph-mon[41654]: osdmap e30: 5 total, 4 up, 5 in 2024-02-05T07:48:16.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:15 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:48:16.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:15 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:48:16.791 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:16 smithi157 ceph-mon[33345]: purged_snaps scrub starts 2024-02-05T07:48:16.791 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:16 smithi157 ceph-mon[33345]: purged_snaps scrub ok 2024-02-05T07:48:16.791 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:16 smithi157 ceph-mon[33345]: osd.4 v2:172.21.15.157:6800/1828659015 boot 2024-02-05T07:48:16.791 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:16 smithi157 ceph-mon[33345]: osdmap e31: 5 total, 5 up, 5 in 2024-02-05T07:48:16.791 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:16 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:48:16.791 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:16 smithi157 ceph-mon[33345]: osdmap e32: 5 total, 5 up, 5 in 2024-02-05T07:48:17.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:16 smithi131 ceph-mon[41654]: purged_snaps scrub starts 2024-02-05T07:48:17.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:16 smithi131 ceph-mon[41654]: purged_snaps scrub ok 2024-02-05T07:48:17.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:16 smithi131 ceph-mon[41654]: osd.4 v2:172.21.15.157:6800/1828659015 boot 2024-02-05T07:48:17.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:16 smithi131 ceph-mon[41654]: osdmap e31: 5 total, 5 up, 5 in 2024-02-05T07:48:17.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:16 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:48:17.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:16 smithi131 ceph-mon[41654]: osdmap e32: 5 total, 5 up, 5 in 2024-02-05T07:48:17.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:16 smithi131 ceph-mon[29507]: purged_snaps scrub starts 2024-02-05T07:48:17.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:16 smithi131 ceph-mon[29507]: purged_snaps scrub ok 2024-02-05T07:48:17.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:16 smithi131 ceph-mon[29507]: osd.4 v2:172.21.15.157:6800/1828659015 boot 2024-02-05T07:48:17.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:16 smithi131 ceph-mon[29507]: osdmap e31: 5 total, 5 up, 5 in 2024-02-05T07:48:17.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:16 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:48:17.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:16 smithi131 ceph-mon[29507]: osdmap e32: 5 total, 5 up, 5 in 2024-02-05T07:48:17.557 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-05T07:48:17.557 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-05T07:48:17.557 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-02-05T07:48:17.557 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-02-05T07:48:17.557 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012327 s, 851 MB/s 2024-02-05T07:48:17.557 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-02-05T07:48:17.907 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:17 smithi157 ceph-mon[33345]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:17.907 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:17 smithi157 ceph-mon[33345]: osdmap e33: 5 total, 5 up, 5 in 2024-02-05T07:48:18.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:17 smithi131 ceph-mon[41654]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:18.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:17 smithi131 ceph-mon[41654]: osdmap e33: 5 total, 5 up, 5 in 2024-02-05T07:48:18.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:17 smithi131 ceph-mon[29507]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:18.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:17 smithi131 ceph-mon[29507]: osdmap e33: 5 total, 5 up, 5 in 2024-02-05T07:48:18.305 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi157:vg_nvme/lv_3 2024-02-05T07:48:18.879 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:48:20.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[29507]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:20.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[29507]: osdmap e34: 5 total, 5 up, 5 in 2024-02-05T07:48:20.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:20.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:20.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:48:20.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:20.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:20.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:48:20.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:20.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:48:20.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:20.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:48:20.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:20.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[41654]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:20.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[41654]: osdmap e34: 5 total, 5 up, 5 in 2024-02-05T07:48:20.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:20.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:20.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:48:20.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:20.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:20.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:48:20.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:20.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:48:20.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:20.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:48:20.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:19 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:20.106 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:19 smithi157 ceph-mon[33345]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:20.106 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:19 smithi157 ceph-mon[33345]: osdmap e34: 5 total, 5 up, 5 in 2024-02-05T07:48:20.106 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:19 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:20.106 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:19 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:20.106 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:19 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:48:20.106 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:19 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:20.106 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:19 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:20.106 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:19 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:48:20.107 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:19 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:20.107 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:19 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:48:20.107 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:19 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:20.107 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:19 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:48:20.107 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:19 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:20.450 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.448+0000 7f1824b3e700 1 -- 172.21.15.157:0/3740535821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820100c80 msgr2=0x7f18201010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:48:20.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.448+0000 7f1824b3e700 1 --2- 172.21.15.157:0/3740535821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820100c80 0x7f18201010a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f18140099c0 tx=0x7f1814009cd0 comp rx=0 tx=0).stop 2024-02-05T07:48:20.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.449+0000 7f1824b3e700 1 -- 172.21.15.157:0/3740535821 shutdown_connections 2024-02-05T07:48:20.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.449+0000 7f1824b3e700 1 --2- 172.21.15.157:0/3740535821 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1820102820 0x7f18201090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:20.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.449+0000 7f1824b3e700 1 --2- 172.21.15.157:0/3740535821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820100c80 0x7f18201010a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:20.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.449+0000 7f1824b3e700 1 --2- 172.21.15.157:0/3740535821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1820101e80 0x7f18201022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:20.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.449+0000 7f1824b3e700 1 -- 172.21.15.157:0/3740535821 >> 172.21.15.157:0/3740535821 conn(0x7f18200fc240 msgr2=0x7f18200fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:48:20.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.449+0000 7f1824b3e700 1 -- 172.21.15.157:0/3740535821 shutdown_connections 2024-02-05T07:48:20.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.449+0000 7f1824b3e700 1 -- 172.21.15.157:0/3740535821 wait complete. 2024-02-05T07:48:20.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.449+0000 7f1824b3e700 1 Processor -- start 2024-02-05T07:48:20.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.449+0000 7f1824b3e700 1 -- start start 2024-02-05T07:48:20.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f1824b3e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1820100c80 0x7f18201963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:48:20.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f1824b3e700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1820101e80 0x7f18201968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:48:20.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f1824b3e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820102820 0x7f182019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:48:20.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f1824b3e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182019dee0 con 0x7f1820100c80 2024-02-05T07:48:20.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f1824b3e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182019e020 con 0x7f1820102820 2024-02-05T07:48:20.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f1824b3e700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182019e1b0 con 0x7f1820101e80 2024-02-05T07:48:20.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f181ed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820102820 0x7f182019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:48:20.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f181ed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820102820 0x7f182019d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49114/0 (socket says 172.21.15.157:49114) 2024-02-05T07:48:20.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f181ed9d700 1 -- 172.21.15.157:0/255728765 learned_addr learned my addr 172.21.15.157:0/255728765 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:48:20.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f181dd9b700 1 --2- 172.21.15.157:0/255728765 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1820101e80 0x7f18201968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:48:20.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f181e59c700 1 --2- 172.21.15.157:0/255728765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1820100c80 0x7f18201963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:48:20.454 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f181ed9d700 1 -- 172.21.15.157:0/255728765 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1820101e80 msgr2=0x7f18201968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:48:20.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f181ed9d700 1 --2- 172.21.15.157:0/255728765 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1820101e80 0x7f18201968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:20.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f181ed9d700 1 -- 172.21.15.157:0/255728765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1820100c80 msgr2=0x7f18201963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:48:20.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f181ed9d700 1 --2- 172.21.15.157:0/255728765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1820100c80 0x7f18201963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:20.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f181ed9d700 1 -- 172.21.15.157:0/255728765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18140095f0 con 0x7f1820102820 2024-02-05T07:48:20.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f181ed9d700 1 --2- 172.21.15.157:0/255728765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820102820 0x7f182019d910 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1818007d70 tx=0x7f181800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:48:20.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.450+0000 7f180f7fe700 1 -- 172.21.15.157:0/255728765 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181800ac90 con 0x7f1820102820 2024-02-05T07:48:20.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.451+0000 7f1824b3e700 1 -- 172.21.15.157:0/255728765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182019e490 con 0x7f1820102820 2024-02-05T07:48:20.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.451+0000 7f180f7fe700 1 -- 172.21.15.157:0/255728765 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f181800adf0 con 0x7f1820102820 2024-02-05T07:48:20.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.451+0000 7f180f7fe700 1 -- 172.21.15.157:0/255728765 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18180115e0 con 0x7f1820102820 2024-02-05T07:48:20.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.451+0000 7f1824b3e700 1 -- 172.21.15.157:0/255728765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182019e940 con 0x7f1820102820 2024-02-05T07:48:20.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.452+0000 7f1824b3e700 1 -- 172.21.15.157:0/255728765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1820107860 con 0x7f1820102820 2024-02-05T07:48:20.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.452+0000 7f180f7fe700 1 -- 172.21.15.157:0/255728765 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f1818012420 con 0x7f1820102820 2024-02-05T07:48:20.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.452+0000 7f180f7fe700 1 --2- 172.21.15.157:0/255728765 >> v2:172.21.15.131:6800/760045032 conn(0x7f180806d990 0x7f180806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:48:20.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.453+0000 7f180f7fe700 1 -- 172.21.15.157:0/255728765 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 3205+0+0 (secure 0 0 0) 0x7f181808c8c0 con 0x7f1820102820 2024-02-05T07:48:20.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.453+0000 7f181e59c700 1 --2- 172.21.15.157:0/255728765 >> v2:172.21.15.131:6800/760045032 conn(0x7f180806d990 0x7f180806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:48:20.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.453+0000 7f181e59c700 1 --2- 172.21.15.157:0/255728765 >> v2:172.21.15.131:6800/760045032 conn(0x7f180806d990 0x7f180806fe50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1814006010 tx=0x7f1814005060 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:48:20.459 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.457+0000 7f180f7fe700 1 -- 172.21.15.157:0/255728765 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1818056da0 con 0x7f1820102820 2024-02-05T07:48:20.661 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:20.659+0000 7f1824b3e700 1 -- 172.21.15.157:0/255728765 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f182010c000 con 0x7f180806d990 2024-02-05T07:48:21.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:20 smithi131 ceph-mon[29507]: Detected new or changed devices on smithi157 2024-02-05T07:48:21.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:20 smithi131 ceph-mon[29507]: Adjusting osd_memory_target on smithi157 to 17088M 2024-02-05T07:48:21.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:20 smithi131 ceph-mon[29507]: osdmap e35: 5 total, 5 up, 5 in 2024-02-05T07:48:21.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:20 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:48:21.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:20 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:48:21.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:20 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:21.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:20 smithi131 ceph-mon[41654]: Detected new or changed devices on smithi157 2024-02-05T07:48:21.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:20 smithi131 ceph-mon[41654]: Adjusting osd_memory_target on smithi157 to 17088M 2024-02-05T07:48:21.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:20 smithi131 ceph-mon[41654]: osdmap e35: 5 total, 5 up, 5 in 2024-02-05T07:48:21.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:20 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:48:21.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:20 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:48:21.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:20 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:20 smithi157 ceph-mon[33345]: Detected new or changed devices on smithi157 2024-02-05T07:48:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:20 smithi157 ceph-mon[33345]: Adjusting osd_memory_target on smithi157 to 17088M 2024-02-05T07:48:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:20 smithi157 ceph-mon[33345]: osdmap e35: 5 total, 5 up, 5 in 2024-02-05T07:48:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:20 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:48:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:20 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:48:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:20 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:21.874 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:21 smithi157 ceph-mon[33345]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:21.874 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:21 smithi157 ceph-mon[33345]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:48:22.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:21 smithi131 ceph-mon[41654]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:22.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:21 smithi131 ceph-mon[41654]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:48:22.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:21 smithi131 ceph-mon[29507]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:22.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:21 smithi131 ceph-mon[29507]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:48:23.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:23 smithi157 ceph-mon[33345]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-05T07:48:23.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.157:0/3032247986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96c3644d-9b3a-4826-befe-0f6cc4af9baf"}]: dispatch 2024-02-05T07:48:23.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96c3644d-9b3a-4826-befe-0f6cc4af9baf"}]: dispatch 2024-02-05T07:48:23.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96c3644d-9b3a-4826-befe-0f6cc4af9baf"}]': finished 2024-02-05T07:48:23.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:23 smithi157 ceph-mon[33345]: osdmap e36: 6 total, 5 up, 6 in 2024-02-05T07:48:23.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:23 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:24.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:23 smithi131 ceph-mon[41654]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-05T07:48:24.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.157:0/3032247986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96c3644d-9b3a-4826-befe-0f6cc4af9baf"}]: dispatch 2024-02-05T07:48:24.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96c3644d-9b3a-4826-befe-0f6cc4af9baf"}]: dispatch 2024-02-05T07:48:24.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96c3644d-9b3a-4826-befe-0f6cc4af9baf"}]': finished 2024-02-05T07:48:24.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:23 smithi131 ceph-mon[41654]: osdmap e36: 6 total, 5 up, 6 in 2024-02-05T07:48:24.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:23 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:24.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:23 smithi131 ceph-mon[29507]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-05T07:48:24.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.157:0/3032247986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96c3644d-9b3a-4826-befe-0f6cc4af9baf"}]: dispatch 2024-02-05T07:48:24.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96c3644d-9b3a-4826-befe-0f6cc4af9baf"}]: dispatch 2024-02-05T07:48:24.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96c3644d-9b3a-4826-befe-0f6cc4af9baf"}]': finished 2024-02-05T07:48:24.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:23 smithi131 ceph-mon[29507]: osdmap e36: 6 total, 5 up, 6 in 2024-02-05T07:48:24.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:23 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:25.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:24 smithi131 ceph-mon[41654]: from='client.? 172.21.15.157:0/3922471264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:48:25.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.157:0/3922471264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:48:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:24 smithi157 ceph-mon[33345]: from='client.? 172.21.15.157:0/3922471264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:48:26.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:25 smithi131 ceph-mon[29507]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-05T07:48:26.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:25 smithi131 ceph-mon[41654]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-05T07:48:26.193 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:25 smithi157 ceph-mon[33345]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-05T07:48:27.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:27 smithi157 ceph-mon[33345]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-02-05T07:48:28.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:27 smithi131 ceph-mon[29507]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-02-05T07:48:28.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:27 smithi131 ceph-mon[41654]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-02-05T07:48:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:28 smithi157 ceph-mon[33345]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-02-05T07:48:29.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:28 smithi131 ceph-mon[29507]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-02-05T07:48:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:28 smithi131 ceph-mon[41654]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-02-05T07:48:29.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:29 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-05T07:48:29.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:29 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:29.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:29 smithi131 ceph-mon[29507]: Deploying daemon osd.5 on smithi157 2024-02-05T07:48:30.156 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:29 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-05T07:48:30.156 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:29 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:30.156 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:29 smithi157 ceph-mon[33345]: Deploying daemon osd.5 on smithi157 2024-02-05T07:48:30.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:29 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-05T07:48:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:29 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:29 smithi131 ceph-mon[41654]: Deploying daemon osd.5 on smithi157 2024-02-05T07:48:31.194 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:30 smithi157 ceph-mon[33345]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-05T07:48:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:30 smithi131 ceph-mon[41654]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-05T07:48:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:30 smithi131 ceph-mon[29507]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-05T07:48:33.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:33 smithi157 ceph-mon[33345]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:34.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:33 smithi131 ceph-mon[41654]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:34.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:33 smithi131 ceph-mon[29507]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:35.814 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:35 smithi157 ceph-mon[33345]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:35.814 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:35 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:35.814 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:35 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:35.814 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:35 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:48:36.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:35 smithi131 ceph-mon[41654]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:35 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:35 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:35 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:48:36.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:35 smithi131 ceph-mon[29507]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:36.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:36.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:36.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:48:37.824 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:37 smithi157 ceph-mon[33345]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:38.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:37 smithi131 ceph-mon[41654]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:38.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:37 smithi131 ceph-mon[29507]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:39.173 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:38 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:39.174 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:38 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:39.174 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:38 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:39.174 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:38 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:48:39.174 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:38 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:39.174 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:38 smithi157 ceph-mon[33345]: from='osd.5 v2:172.21.15.157:6804/3589104149' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-05T07:48:39.174 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:38 smithi157 ceph-mon[33345]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-05T07:48:39.174 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:38 smithi157 ceph-mon[33345]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:39.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:39.172+0000 7f180f7fe700 1 -- 172.21.15.157:0/255728765 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f182010c000 con 0x7f180806d990 2024-02-05T07:48:39.175 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 5 on host 'smithi157' 2024-02-05T07:48:39.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:39.175+0000 7f1824b3e700 1 -- 172.21.15.157:0/255728765 >> v2:172.21.15.131:6800/760045032 conn(0x7f180806d990 msgr2=0x7f180806fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:48:39.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:39.175+0000 7f1824b3e700 1 --2- 172.21.15.157:0/255728765 >> v2:172.21.15.131:6800/760045032 conn(0x7f180806d990 0x7f180806fe50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1814006010 tx=0x7f1814005060 comp rx=0 tx=0).stop 2024-02-05T07:48:39.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:39.175+0000 7f1824b3e700 1 -- 172.21.15.157:0/255728765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820102820 msgr2=0x7f182019d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:48:39.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:39.175+0000 7f1824b3e700 1 --2- 172.21.15.157:0/255728765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820102820 0x7f182019d910 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1818007d70 tx=0x7f181800a340 comp rx=0 tx=0).stop 2024-02-05T07:48:39.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:39.175+0000 7f1824b3e700 1 -- 172.21.15.157:0/255728765 shutdown_connections 2024-02-05T07:48:39.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:39.175+0000 7f1824b3e700 1 --2- 172.21.15.157:0/255728765 >> v2:172.21.15.131:6800/760045032 conn(0x7f180806d990 0x7f180806fe50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:39.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:39.175+0000 7f1824b3e700 1 --2- 172.21.15.157:0/255728765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1820100c80 0x7f18201963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:39.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:39.175+0000 7f1824b3e700 1 --2- 172.21.15.157:0/255728765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820102820 0x7f182019d910 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:39.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:39.175+0000 7f1824b3e700 1 --2- 172.21.15.157:0/255728765 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1820101e80 0x7f18201968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:39.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:39.175+0000 7f1824b3e700 1 -- 172.21.15.157:0/255728765 >> 172.21.15.157:0/255728765 conn(0x7f18200fc240 msgr2=0x7f18200fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:48:39.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:39.175+0000 7f1824b3e700 1 -- 172.21.15.157:0/255728765 shutdown_connections 2024-02-05T07:48:39.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:39.175+0000 7f1824b3e700 1 -- 172.21.15.157:0/255728765 wait complete. 2024-02-05T07:48:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:38 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:38 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:38 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:38 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:48:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:38 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:38 smithi131 ceph-mon[41654]: from='osd.5 v2:172.21.15.157:6804/3589104149' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-05T07:48:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:38 smithi131 ceph-mon[41654]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-05T07:48:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:38 smithi131 ceph-mon[41654]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:39.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:38 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:39.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:38 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:39.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:38 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:39.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:38 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:48:39.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:38 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:39.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:38 smithi131 ceph-mon[29507]: from='osd.5 v2:172.21.15.157:6804/3589104149' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-05T07:48:39.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:38 smithi131 ceph-mon[29507]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-05T07:48:39.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:38 smithi131 ceph-mon[29507]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:39.697 DEBUG:teuthology.orchestra.run.smithi157:osd.5> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.5.service 2024-02-05T07:48:39.699 INFO:tasks.cephadm:Deploying osd.6 on smithi157 with /dev/vg_nvme/lv_2... 2024-02-05T07:48:39.699 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-05T07:48:39.967 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:39 smithi157 ceph-mon[33345]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-05T07:48:39.967 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:39 smithi157 ceph-mon[33345]: from='osd.5 v2:172.21.15.157:6804/3589104149' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:48:39.967 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:39 smithi157 ceph-mon[33345]: osdmap e37: 6 total, 5 up, 6 in 2024-02-05T07:48:39.967 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:39 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:39.967 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:39 smithi157 ceph-mon[33345]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:48:39.967 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:39 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:39.967 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:39 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:39.967 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:39 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:48:40.221 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 07:48:40 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T07:48:40.172+0000 7f867c74a700 -1 osd.5 0 waiting for initial osdmap 2024-02-05T07:48:40.221 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 07:48:40 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T07:48:40.176+0000 7f8673d3b700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T07:48:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:39 smithi131 ceph-mon[41654]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-05T07:48:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:39 smithi131 ceph-mon[41654]: from='osd.5 v2:172.21.15.157:6804/3589104149' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:48:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:39 smithi131 ceph-mon[41654]: osdmap e37: 6 total, 5 up, 6 in 2024-02-05T07:48:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:39 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:39 smithi131 ceph-mon[41654]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:48:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:39 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:39 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:39 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:48:40.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:39 smithi131 ceph-mon[29507]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-05T07:48:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:39 smithi131 ceph-mon[29507]: from='osd.5 v2:172.21.15.157:6804/3589104149' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:48:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:39 smithi131 ceph-mon[29507]: osdmap e37: 6 total, 5 up, 6 in 2024-02-05T07:48:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:39 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:39 smithi131 ceph-mon[29507]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:48:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:39 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:39 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:39 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:48:40.454 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:48:41.214 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:40 smithi157 ceph-mon[33345]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-02-05T07:48:41.214 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:40 smithi157 ceph-mon[33345]: osdmap e38: 6 total, 5 up, 6 in 2024-02-05T07:48:41.214 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:41.214 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:41.214 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:40 smithi157 ceph-mon[33345]: from='osd.5 ' entity='osd.5' 2024-02-05T07:48:41.214 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:40 smithi157 ceph-mon[33345]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:41.214 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:40 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:40 smithi131 ceph-mon[41654]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-02-05T07:48:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:40 smithi131 ceph-mon[41654]: osdmap e38: 6 total, 5 up, 6 in 2024-02-05T07:48:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:40 smithi131 ceph-mon[41654]: from='osd.5 ' entity='osd.5' 2024-02-05T07:48:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:40 smithi131 ceph-mon[41654]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:40 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:41.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:40 smithi131 ceph-mon[29507]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-02-05T07:48:41.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:40 smithi131 ceph-mon[29507]: osdmap e38: 6 total, 5 up, 6 in 2024-02-05T07:48:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:40 smithi131 ceph-mon[29507]: from='osd.5 ' entity='osd.5' 2024-02-05T07:48:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:40 smithi131 ceph-mon[29507]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:48:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:40 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:41 smithi157 ceph-mon[33345]: purged_snaps scrub starts 2024-02-05T07:48:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:41 smithi157 ceph-mon[33345]: purged_snaps scrub ok 2024-02-05T07:48:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:41 smithi157 ceph-mon[33345]: osd.5 v2:172.21.15.157:6804/3589104149 boot 2024-02-05T07:48:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:41 smithi157 ceph-mon[33345]: osdmap e39: 6 total, 6 up, 6 in 2024-02-05T07:48:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:41 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:41 smithi131 ceph-mon[29507]: purged_snaps scrub starts 2024-02-05T07:48:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:41 smithi131 ceph-mon[29507]: purged_snaps scrub ok 2024-02-05T07:48:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:41 smithi131 ceph-mon[29507]: osd.5 v2:172.21.15.157:6804/3589104149 boot 2024-02-05T07:48:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:41 smithi131 ceph-mon[29507]: osdmap e39: 6 total, 6 up, 6 in 2024-02-05T07:48:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:41 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:42.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:41 smithi131 ceph-mon[41654]: purged_snaps scrub starts 2024-02-05T07:48:42.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:41 smithi131 ceph-mon[41654]: purged_snaps scrub ok 2024-02-05T07:48:42.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:41 smithi131 ceph-mon[41654]: osd.5 v2:172.21.15.157:6804/3589104149 boot 2024-02-05T07:48:42.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:41 smithi131 ceph-mon[41654]: osdmap e39: 6 total, 6 up, 6 in 2024-02-05T07:48:42.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:41 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:48:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:42 smithi157 ceph-mon[33345]: osdmap e40: 6 total, 6 up, 6 in 2024-02-05T07:48:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:42 smithi157 ceph-mon[33345]: pgmap v136: 1 pgs: 1 remapped; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:48:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:42 smithi157 ceph-mon[33345]: osdmap e41: 6 total, 6 up, 6 in 2024-02-05T07:48:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:42 smithi131 ceph-mon[29507]: osdmap e40: 6 total, 6 up, 6 in 2024-02-05T07:48:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:42 smithi131 ceph-mon[29507]: pgmap v136: 1 pgs: 1 remapped; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:48:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:42 smithi131 ceph-mon[29507]: osdmap e41: 6 total, 6 up, 6 in 2024-02-05T07:48:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:42 smithi131 ceph-mon[41654]: osdmap e40: 6 total, 6 up, 6 in 2024-02-05T07:48:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:42 smithi131 ceph-mon[41654]: pgmap v136: 1 pgs: 1 remapped; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:48:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:42 smithi131 ceph-mon[41654]: osdmap e41: 6 total, 6 up, 6 in 2024-02-05T07:48:43.815 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-05T07:48:43.815 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-05T07:48:43.815 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-02-05T07:48:43.815 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-02-05T07:48:43.815 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00867398 s, 1.2 GB/s 2024-02-05T07:48:43.815 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-02-05T07:48:44.444 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:44 smithi157 ceph-mon[33345]: Detected new or changed devices on smithi157 2024-02-05T07:48:44.444 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:44 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:44.444 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:44 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:44.444 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:44 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:48:44.444 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:44 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:48:44.444 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:44 smithi157 ceph-mon[33345]: Adjusting osd_memory_target on smithi157 to 8544M 2024-02-05T07:48:44.444 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:44 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:44.444 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:44 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:44.444 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:44 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:48:44.444 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:44 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:44.444 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:44 smithi157 ceph-mon[33345]: osdmap e42: 6 total, 6 up, 6 in 2024-02-05T07:48:44.472 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi157:vg_nvme/lv_2 2024-02-05T07:48:44.707 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:48:44.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[41654]: Detected new or changed devices on smithi157 2024-02-05T07:48:44.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:44.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:44.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:48:44.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:48:44.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[41654]: Adjusting osd_memory_target on smithi157 to 8544M 2024-02-05T07:48:44.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:44.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:44.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:48:44.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:44.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[41654]: osdmap e42: 6 total, 6 up, 6 in 2024-02-05T07:48:44.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[29507]: Detected new or changed devices on smithi157 2024-02-05T07:48:44.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:44.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:44.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:48:44.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:48:44.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[29507]: Adjusting osd_memory_target on smithi157 to 8544M 2024-02-05T07:48:44.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:44.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:44.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:48:44.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:48:44.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:44 smithi131 ceph-mon[29507]: osdmap e42: 6 total, 6 up, 6 in 2024-02-05T07:48:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:45 smithi157 ceph-mon[33345]: pgmap v139: 1 pgs: 1 remapped; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:48:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:45 smithi157 ceph-mon[33345]: osdmap e43: 6 total, 6 up, 6 in 2024-02-05T07:48:45.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:45 smithi131 ceph-mon[29507]: pgmap v139: 1 pgs: 1 remapped; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:48:45.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:45 smithi131 ceph-mon[29507]: osdmap e43: 6 total, 6 up, 6 in 2024-02-05T07:48:45.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:45 smithi131 ceph-mon[41654]: pgmap v139: 1 pgs: 1 remapped; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:48:45.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:45 smithi131 ceph-mon[41654]: osdmap e43: 6 total, 6 up, 6 in 2024-02-05T07:48:46.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.306+0000 7f760332a700 1 -- 172.21.15.157:0/1307634846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75fc100c50 msgr2=0x7f75fc101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:48:46.309 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.306+0000 7f760332a700 1 --2- 172.21.15.157:0/1307634846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75fc100c50 0x7f75fc101070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f75f00099c0 tx=0x7f75f0009cd0 comp rx=0 tx=0).stop 2024-02-05T07:48:46.309 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.306+0000 7f760332a700 1 -- 172.21.15.157:0/1307634846 shutdown_connections 2024-02-05T07:48:46.309 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.306+0000 7f760332a700 1 --2- 172.21.15.157:0/1307634846 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f75fc1027f0 0x7f75fc1090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:46.309 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.306+0000 7f760332a700 1 --2- 172.21.15.157:0/1307634846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75fc100c50 0x7f75fc101070 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:46.309 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.306+0000 7f760332a700 1 --2- 172.21.15.157:0/1307634846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75fc101e50 0x7f75fc1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:46.309 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.306+0000 7f760332a700 1 -- 172.21.15.157:0/1307634846 >> 172.21.15.157:0/1307634846 conn(0x7f75fc0fc1f0 msgr2=0x7f75fc0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:48:46.309 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.306+0000 7f760332a700 1 -- 172.21.15.157:0/1307634846 shutdown_connections 2024-02-05T07:48:46.309 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.306+0000 7f760332a700 1 -- 172.21.15.157:0/1307634846 wait complete. 2024-02-05T07:48:46.309 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f760332a700 1 Processor -- start 2024-02-05T07:48:46.309 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f760332a700 1 -- start start 2024-02-05T07:48:46.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f760332a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75fc100c50 0x7f75fc104be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:48:46.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f760332a700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f75fc101e50 0x7f75fc103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:48:46.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f760332a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75fc1027f0 0x7f75fc103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:48:46.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f760332a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc103ef0 con 0x7f75fc1027f0 2024-02-05T07:48:46.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f760332a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc104060 con 0x7f75fc100c50 2024-02-05T07:48:46.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f760332a700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc1a1590 con 0x7f75fc101e50 2024-02-05T07:48:46.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f76010c6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75fc100c50 0x7f75fc104be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:48:46.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f76010c6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75fc100c50 0x7f75fc104be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54468/0 (socket says 172.21.15.157:54468) 2024-02-05T07:48:46.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f76010c6700 1 -- 172.21.15.157:0/1222410910 learned_addr learned my addr 172.21.15.157:0/1222410910 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:48:46.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f76008c5700 1 --2- 172.21.15.157:0/1222410910 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f75fc101e50 0x7f75fc103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:48:46.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f76010c6700 1 -- 172.21.15.157:0/1222410910 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f75fc101e50 msgr2=0x7f75fc103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:48:46.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f76010c6700 1 --2- 172.21.15.157:0/1222410910 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f75fc101e50 0x7f75fc103230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:46.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f76010c6700 1 -- 172.21.15.157:0/1222410910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75fc1027f0 msgr2=0x7f75fc103770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:48:46.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f76018c7700 1 --2- 172.21.15.157:0/1222410910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75fc1027f0 0x7f75fc103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:48:46.310 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f76010c6700 1 --2- 172.21.15.157:0/1222410910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75fc1027f0 0x7f75fc103770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:48:46.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f76010c6700 1 -- 172.21.15.157:0/1222410910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f00095f0 con 0x7f75fc100c50 2024-02-05T07:48:46.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.307+0000 7f76010c6700 1 --2- 172.21.15.157:0/1222410910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75fc100c50 0x7f75fc104be0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009f50 tx=0x7f75f00038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:48:46.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.308+0000 7f75ee7fc700 1 -- 172.21.15.157:0/1222410910 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f001d070 con 0x7f75fc100c50 2024-02-05T07:48:46.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.308+0000 7f75ee7fc700 1 -- 172.21.15.157:0/1222410910 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f75f00041c0 con 0x7f75fc100c50 2024-02-05T07:48:46.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.308+0000 7f760332a700 1 -- 172.21.15.157:0/1222410910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75fc1a1810 con 0x7f75fc100c50 2024-02-05T07:48:46.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.308+0000 7f75ee7fc700 1 -- 172.21.15.157:0/1222410910 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f000f460 con 0x7f75fc100c50 2024-02-05T07:48:46.311 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.308+0000 7f760332a700 1 -- 172.21.15.157:0/1222410910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75fc1a1ca0 con 0x7f75fc100c50 2024-02-05T07:48:46.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.309+0000 7f760332a700 1 -- 172.21.15.157:0/1222410910 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75fc108a30 con 0x7f75fc100c50 2024-02-05T07:48:46.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.309+0000 7f75ee7fc700 1 -- 172.21.15.157:0/1222410910 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f75f0022670 con 0x7f75fc100c50 2024-02-05T07:48:46.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.309+0000 7f75ee7fc700 1 --2- 172.21.15.157:0/1222410910 >> v2:172.21.15.131:6800/760045032 conn(0x7f75e806d990 0x7f75e806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:48:46.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.309+0000 7f75ee7fc700 1 -- 172.21.15.157:0/1222410910 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 3497+0+0 (secure 0 0 0) 0x7f75f008dd80 con 0x7f75fc100c50 2024-02-05T07:48:46.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.310+0000 7f76008c5700 1 --2- 172.21.15.157:0/1222410910 >> v2:172.21.15.131:6800/760045032 conn(0x7f75e806d990 0x7f75e806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:48:46.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.310+0000 7f76008c5700 1 --2- 172.21.15.157:0/1222410910 >> v2:172.21.15.131:6800/760045032 conn(0x7f75e806d990 0x7f75e806fe50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f75f8006fd0 tx=0x7f75f8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:48:46.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.314+0000 7f75ee7fc700 1 -- 172.21.15.157:0/1222410910 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f75f0058140 con 0x7f75fc100c50 2024-02-05T07:48:46.517 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:48:46.516+0000 7f760332a700 1 -- 172.21.15.157:0/1222410910 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f75fc10bf90 con 0x7f75e806d990 2024-02-05T07:48:46.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:46 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:48:46.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:46 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:48:46.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:46 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:47.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:46 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:48:47.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:46 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:48:47.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:46 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:47.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:46 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:48:47.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:46 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:48:47.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:46 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:47.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:47 smithi157 ceph-mon[33345]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:48:47.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:47 smithi157 ceph-mon[33345]: pgmap v141: 1 pgs: 1 remapped; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:48:48.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:47 smithi131 ceph-mon[29507]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:48:48.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:47 smithi131 ceph-mon[29507]: pgmap v141: 1 pgs: 1 remapped; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:48:48.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:47 smithi131 ceph-mon[41654]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:48:48.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:47 smithi131 ceph-mon[41654]: pgmap v141: 1 pgs: 1 remapped; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:48:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:49 smithi157 ceph-mon[33345]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-05T07:48:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.157:0/870640720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24119b57-55ed-4b51-b25e-7a6cea556af3"}]: dispatch 2024-02-05T07:48:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24119b57-55ed-4b51-b25e-7a6cea556af3"}]: dispatch 2024-02-05T07:48:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24119b57-55ed-4b51-b25e-7a6cea556af3"}]': finished 2024-02-05T07:48:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:49 smithi157 ceph-mon[33345]: osdmap e44: 7 total, 6 up, 7 in 2024-02-05T07:48:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:49 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:48:50.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:49 smithi131 ceph-mon[41654]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-05T07:48:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.157:0/870640720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24119b57-55ed-4b51-b25e-7a6cea556af3"}]: dispatch 2024-02-05T07:48:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24119b57-55ed-4b51-b25e-7a6cea556af3"}]: dispatch 2024-02-05T07:48:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24119b57-55ed-4b51-b25e-7a6cea556af3"}]': finished 2024-02-05T07:48:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:49 smithi131 ceph-mon[41654]: osdmap e44: 7 total, 6 up, 7 in 2024-02-05T07:48:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:49 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:48:50.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:49 smithi131 ceph-mon[29507]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-05T07:48:50.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.157:0/870640720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24119b57-55ed-4b51-b25e-7a6cea556af3"}]: dispatch 2024-02-05T07:48:50.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24119b57-55ed-4b51-b25e-7a6cea556af3"}]: dispatch 2024-02-05T07:48:50.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24119b57-55ed-4b51-b25e-7a6cea556af3"}]': finished 2024-02-05T07:48:50.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:49 smithi131 ceph-mon[29507]: osdmap e44: 7 total, 6 up, 7 in 2024-02-05T07:48:50.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:49 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:48:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.157:0/2246272300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:48:51.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.157:0/2246272300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:48:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.157:0/2246272300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:48:51.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:51 smithi157 ceph-mon[33345]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-02-05T07:48:52.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:51 smithi131 ceph-mon[29507]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-02-05T07:48:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:51 smithi131 ceph-mon[41654]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-02-05T07:48:53.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:53 smithi157 ceph-mon[33345]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-05T07:48:54.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:53 smithi131 ceph-mon[29507]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-05T07:48:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:53 smithi131 ceph-mon[41654]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-05T07:48:55.822 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:55 smithi157 ceph-mon[33345]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-05T07:48:56.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:55 smithi131 ceph-mon[41654]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-05T07:48:56.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:55 smithi131 ceph-mon[29507]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-05T07:48:57.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:56 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-05T07:48:57.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:56 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:57.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-05T07:48:57.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:57.203 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:56 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-05T07:48:57.203 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:56 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:57.908 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:57 smithi157 ceph-mon[33345]: Deploying daemon osd.6 on smithi157 2024-02-05T07:48:57.908 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:57 smithi157 ceph-mon[33345]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-05T07:48:58.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:57 smithi131 ceph-mon[29507]: Deploying daemon osd.6 on smithi157 2024-02-05T07:48:58.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:57 smithi131 ceph-mon[29507]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-05T07:48:58.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:57 smithi131 ceph-mon[41654]: Deploying daemon osd.6 on smithi157 2024-02-05T07:48:58.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:57 smithi131 ceph-mon[41654]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-05T07:48:59.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:48:59 smithi157 ceph-mon[33345]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:49:00.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:48:59 smithi131 ceph-mon[29507]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:49:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:48:59 smithi131 ceph-mon[41654]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:49:02.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:01 smithi131 ceph-mon[29507]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:49:02.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:02.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:02.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:01 smithi131 ceph-mon[41654]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:49:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:02.152 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:01 smithi157 ceph-mon[33345]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:49:02.152 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:02.152 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:02.152 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:04.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:03 smithi131 ceph-mon[29507]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:49:04.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:03 smithi131 ceph-mon[41654]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:49:04.210 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:03 smithi157 ceph-mon[33345]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:49:05.282 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:05.280+0000 7f75ee7fc700 1 -- 172.21.15.157:0/1222410910 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f75fc10bf90 con 0x7f75e806d990 2024-02-05T07:49:05.282 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 6 on host 'smithi157' 2024-02-05T07:49:05.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:05.282+0000 7f760332a700 1 -- 172.21.15.157:0/1222410910 >> v2:172.21.15.131:6800/760045032 conn(0x7f75e806d990 msgr2=0x7f75e806fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:05.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:05.282+0000 7f760332a700 1 --2- 172.21.15.157:0/1222410910 >> v2:172.21.15.131:6800/760045032 conn(0x7f75e806d990 0x7f75e806fe50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f75f8006fd0 tx=0x7f75f8008040 comp rx=0 tx=0).stop 2024-02-05T07:49:05.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:05.282+0000 7f760332a700 1 -- 172.21.15.157:0/1222410910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75fc100c50 msgr2=0x7f75fc104be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:05.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:05.282+0000 7f760332a700 1 --2- 172.21.15.157:0/1222410910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75fc100c50 0x7f75fc104be0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009f50 tx=0x7f75f00038a0 comp rx=0 tx=0).stop 2024-02-05T07:49:05.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:05.283+0000 7f760332a700 1 -- 172.21.15.157:0/1222410910 shutdown_connections 2024-02-05T07:49:05.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:05.283+0000 7f760332a700 1 --2- 172.21.15.157:0/1222410910 >> v2:172.21.15.131:6800/760045032 conn(0x7f75e806d990 0x7f75e806fe50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:05.285 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:05.283+0000 7f760332a700 1 --2- 172.21.15.157:0/1222410910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75fc1027f0 0x7f75fc103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:05.285 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:05.283+0000 7f760332a700 1 --2- 172.21.15.157:0/1222410910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75fc100c50 0x7f75fc104be0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:05.285 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:05.283+0000 7f760332a700 1 --2- 172.21.15.157:0/1222410910 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f75fc101e50 0x7f75fc103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:05.285 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:05.283+0000 7f760332a700 1 -- 172.21.15.157:0/1222410910 >> 172.21.15.157:0/1222410910 conn(0x7f75fc0fc1f0 msgr2=0x7f75fc06a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:05.285 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:05.283+0000 7f760332a700 1 -- 172.21.15.157:0/1222410910 shutdown_connections 2024-02-05T07:49:05.285 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:05.283+0000 7f760332a700 1 -- 172.21.15.157:0/1222410910 wait complete. 2024-02-05T07:49:05.925 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:05.925 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:05.925 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:05 smithi157 ceph-mon[33345]: from='osd.6 v2:172.21.15.157:6808/204226792' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-05T07:49:05.925 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:05 smithi157 ceph-mon[33345]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-05T07:49:05.925 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:05 smithi157 ceph-mon[33345]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:49:05.925 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:05.925 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:05.953 DEBUG:teuthology.orchestra.run.smithi157:osd.6> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.6.service 2024-02-05T07:49:05.955 INFO:tasks.cephadm:Deploying osd.7 on smithi157 with /dev/vg_nvme/lv_1... 2024-02-05T07:49:05.956 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-05T07:49:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:05 smithi131 ceph-mon[41654]: from='osd.6 v2:172.21.15.157:6808/204226792' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-05T07:49:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:05 smithi131 ceph-mon[41654]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-05T07:49:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:05 smithi131 ceph-mon[41654]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:49:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:06.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:06.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:06.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:05 smithi131 ceph-mon[29507]: from='osd.6 v2:172.21.15.157:6808/204226792' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-05T07:49:06.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:05 smithi131 ceph-mon[29507]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-05T07:49:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:05 smithi131 ceph-mon[29507]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:49:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:06.840 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:06 smithi157 ceph-mon[33345]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-05T07:49:06.841 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:06 smithi157 ceph-mon[33345]: osdmap e45: 7 total, 6 up, 7 in 2024-02-05T07:49:06.841 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:06 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:49:06.841 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:06 smithi157 ceph-mon[33345]: from='osd.6 v2:172.21.15.157:6808/204226792' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:49:06.841 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:06 smithi157 ceph-mon[33345]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:49:07.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:06 smithi131 ceph-mon[29507]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-05T07:49:07.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:06 smithi131 ceph-mon[29507]: osdmap e45: 7 total, 6 up, 7 in 2024-02-05T07:49:07.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:06 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:49:07.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:06 smithi131 ceph-mon[29507]: from='osd.6 v2:172.21.15.157:6808/204226792' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:49:07.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:06 smithi131 ceph-mon[29507]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:49:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:06 smithi131 ceph-mon[41654]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-05T07:49:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:06 smithi131 ceph-mon[41654]: osdmap e45: 7 total, 6 up, 7 in 2024-02-05T07:49:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:06 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:49:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:06 smithi131 ceph-mon[41654]: from='osd.6 v2:172.21.15.157:6808/204226792' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:49:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:06 smithi131 ceph-mon[41654]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:49:07.092 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 07:49:06 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T07:49:06.838+0000 7f25f123c700 -1 osd.6 0 waiting for initial osdmap 2024-02-05T07:49:07.092 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 07:49:06 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T07:49:06.844+0000 7f25e882d700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T07:49:07.094 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:49:07.695 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:07 smithi157 ceph-mon[33345]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-02-05T07:49:07.695 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:07 smithi157 ceph-mon[33345]: osdmap e46: 7 total, 6 up, 7 in 2024-02-05T07:49:07.695 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:07 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:49:07.695 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:07 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:49:07.695 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:07 smithi157 ceph-mon[33345]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:49:07.695 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:07 smithi157 ceph-mon[33345]: from='osd.6 ' entity='osd.6' 2024-02-05T07:49:08.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:07 smithi131 ceph-mon[29507]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-02-05T07:49:08.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:07 smithi131 ceph-mon[29507]: osdmap e46: 7 total, 6 up, 7 in 2024-02-05T07:49:08.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:07 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:49:08.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:07 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:49:08.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:07 smithi131 ceph-mon[29507]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:49:08.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:07 smithi131 ceph-mon[29507]: from='osd.6 ' entity='osd.6' 2024-02-05T07:49:08.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:07 smithi131 ceph-mon[41654]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-02-05T07:49:08.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:07 smithi131 ceph-mon[41654]: osdmap e46: 7 total, 6 up, 7 in 2024-02-05T07:49:08.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:07 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:49:08.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:07 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:49:08.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:07 smithi131 ceph-mon[41654]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:49:08.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:07 smithi131 ceph-mon[41654]: from='osd.6 ' entity='osd.6' 2024-02-05T07:49:08.920 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:08 smithi157 ceph-mon[33345]: purged_snaps scrub starts 2024-02-05T07:49:08.920 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:08 smithi157 ceph-mon[33345]: purged_snaps scrub ok 2024-02-05T07:49:08.920 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:08 smithi157 ceph-mon[33345]: osd.6 v2:172.21.15.157:6808/204226792 boot 2024-02-05T07:49:08.920 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:08 smithi157 ceph-mon[33345]: osdmap e47: 7 total, 7 up, 7 in 2024-02-05T07:49:08.920 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:08 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:49:08.920 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:08 smithi157 ceph-mon[33345]: osdmap e48: 7 total, 7 up, 7 in 2024-02-05T07:49:09.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:08 smithi131 ceph-mon[41654]: purged_snaps scrub starts 2024-02-05T07:49:09.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:08 smithi131 ceph-mon[41654]: purged_snaps scrub ok 2024-02-05T07:49:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:08 smithi131 ceph-mon[41654]: osd.6 v2:172.21.15.157:6808/204226792 boot 2024-02-05T07:49:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:08 smithi131 ceph-mon[41654]: osdmap e47: 7 total, 7 up, 7 in 2024-02-05T07:49:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:08 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:49:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:08 smithi131 ceph-mon[41654]: osdmap e48: 7 total, 7 up, 7 in 2024-02-05T07:49:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:08 smithi131 ceph-mon[29507]: purged_snaps scrub starts 2024-02-05T07:49:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:08 smithi131 ceph-mon[29507]: purged_snaps scrub ok 2024-02-05T07:49:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:08 smithi131 ceph-mon[29507]: osd.6 v2:172.21.15.157:6808/204226792 boot 2024-02-05T07:49:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:08 smithi131 ceph-mon[29507]: osdmap e47: 7 total, 7 up, 7 in 2024-02-05T07:49:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:08 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:49:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:08 smithi131 ceph-mon[29507]: osdmap e48: 7 total, 7 up, 7 in 2024-02-05T07:49:09.750 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:09 smithi157 ceph-mon[33345]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 625 GiB / 626 GiB avail 2024-02-05T07:49:09.750 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:09 smithi157 ceph-mon[33345]: osdmap e49: 7 total, 7 up, 7 in 2024-02-05T07:49:09.750 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:09 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:09.750 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:09 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:09.750 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:09 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:09.750 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:09 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:09.750 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:09 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:09.750 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:09 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:09.750 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:09 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:09.750 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:09 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:09.750 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:09 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:09.750 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:09 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:09.750 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:09 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:09.751 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:09 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:09.751 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:09 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[41654]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 625 GiB / 626 GiB avail 2024-02-05T07:49:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[41654]: osdmap e49: 7 total, 7 up, 7 in 2024-02-05T07:49:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:10.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:10.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:10.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:10.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:10.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:10.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:10.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:10.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[29507]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 625 GiB / 626 GiB avail 2024-02-05T07:49:10.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[29507]: osdmap e49: 7 total, 7 up, 7 in 2024-02-05T07:49:10.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:10.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:10.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:10.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:10.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:10.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:10.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:10.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:10.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:10.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:10.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:10.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:10.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:09 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:10.081 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-05T07:49:10.082 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-05T07:49:10.082 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-02-05T07:49:10.082 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-02-05T07:49:10.082 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00847201 s, 1.2 GB/s 2024-02-05T07:49:10.082 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-02-05T07:49:10.704 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi157:vg_nvme/lv_1 2024-02-05T07:49:10.938 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:49:10.970 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:10 smithi157 ceph-mon[33345]: Detected new or changed devices on smithi157 2024-02-05T07:49:10.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:10 smithi157 ceph-mon[33345]: Adjusting osd_memory_target on smithi157 to 5696M 2024-02-05T07:49:10.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:10 smithi157 ceph-mon[33345]: osdmap e50: 7 total, 7 up, 7 in 2024-02-05T07:49:11.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:10 smithi131 ceph-mon[41654]: Detected new or changed devices on smithi157 2024-02-05T07:49:11.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:10 smithi131 ceph-mon[41654]: Adjusting osd_memory_target on smithi157 to 5696M 2024-02-05T07:49:11.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:10 smithi131 ceph-mon[41654]: osdmap e50: 7 total, 7 up, 7 in 2024-02-05T07:49:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:10 smithi131 ceph-mon[29507]: Detected new or changed devices on smithi157 2024-02-05T07:49:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:10 smithi131 ceph-mon[29507]: Adjusting osd_memory_target on smithi157 to 5696M 2024-02-05T07:49:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:10 smithi131 ceph-mon[29507]: osdmap e50: 7 total, 7 up, 7 in 2024-02-05T07:49:12.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:11 smithi131 ceph-mon[41654]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 625 GiB / 626 GiB avail 2024-02-05T07:49:12.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:11 smithi131 ceph-mon[41654]: osdmap e51: 7 total, 7 up, 7 in 2024-02-05T07:49:12.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:11 smithi131 ceph-mon[29507]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 625 GiB / 626 GiB avail 2024-02-05T07:49:12.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:11 smithi131 ceph-mon[29507]: osdmap e51: 7 total, 7 up, 7 in 2024-02-05T07:49:12.042 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:11 smithi157 ceph-mon[33345]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 625 GiB / 626 GiB avail 2024-02-05T07:49:12.042 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:11 smithi157 ceph-mon[33345]: osdmap e51: 7 total, 7 up, 7 in 2024-02-05T07:49:12.569 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.567+0000 7f3fd65a9700 1 -- 172.21.15.157:0/2275355803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd0102250 msgr2=0x7f3fd0102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:12.569 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.567+0000 7f3fd65a9700 1 --2- 172.21.15.157:0/2275355803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd0102250 0x7f3fd0102670 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009990 tx=0x7f3fc0009ca0 comp rx=0 tx=0).stop 2024-02-05T07:49:12.569 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.567+0000 7f3fd65a9700 1 -- 172.21.15.157:0/2275355803 shutdown_connections 2024-02-05T07:49:12.569 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.567+0000 7f3fd65a9700 1 --2- 172.21.15.157:0/2275355803 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f3fd0101870 0x7f3fd0109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:12.569 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.567+0000 7f3fd65a9700 1 --2- 172.21.15.157:0/2275355803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd0102250 0x7f3fd0102670 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:12.569 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.567+0000 7f3fd65a9700 1 --2- 172.21.15.157:0/2275355803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fd0100eb0 0x7f3fd0101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:12.569 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.567+0000 7f3fd65a9700 1 -- 172.21.15.157:0/2275355803 >> 172.21.15.157:0/2275355803 conn(0x7f3fd00fb9c0 msgr2=0x7f3fd00fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:12.569 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.567+0000 7f3fd65a9700 1 -- 172.21.15.157:0/2275355803 shutdown_connections 2024-02-05T07:49:12.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.567+0000 7f3fd65a9700 1 -- 172.21.15.157:0/2275355803 wait complete. 2024-02-05T07:49:12.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.567+0000 7f3fd65a9700 1 Processor -- start 2024-02-05T07:49:12.570 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.567+0000 7f3fd65a9700 1 -- start start 2024-02-05T07:49:12.571 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fd65a9700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f3fd0100eb0 0x7f3fd01974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:12.571 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fd65a9700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fd0101870 0x7f3fd0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:12.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fd65a9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd0102250 0x7f3fd019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:12.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fd65a9700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd019f070 con 0x7f3fd0101870 2024-02-05T07:49:12.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fd65a9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd019f1b0 con 0x7f3fd0102250 2024-02-05T07:49:12.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fd65a9700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd019f340 con 0x7f3fd0100eb0 2024-02-05T07:49:12.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fd4b46700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd0102250 0x7f3fd019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:12.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fd4b46700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd0102250 0x7f3fd019eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55478/0 (socket says 172.21.15.157:55478) 2024-02-05T07:49:12.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fd4b46700 1 -- 172.21.15.157:0/119569834 learned_addr learned my addr 172.21.15.157:0/119569834 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:49:12.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fcffff700 1 --2- 172.21.15.157:0/119569834 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f3fd0100eb0 0x7f3fd01974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:12.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fcf7fe700 1 --2- 172.21.15.157:0/119569834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fd0101870 0x7f3fd0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:12.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fd4b46700 1 -- 172.21.15.157:0/119569834 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f3fd0100eb0 msgr2=0x7f3fd01974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:12.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fd4b46700 1 --2- 172.21.15.157:0/119569834 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f3fd0100eb0 0x7f3fd01974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:12.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fd4b46700 1 -- 172.21.15.157:0/119569834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fd0101870 msgr2=0x7f3fd0197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:12.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fd4b46700 1 --2- 172.21.15.157:0/119569834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fd0101870 0x7f3fd0197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:12.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fd4b46700 1 -- 172.21.15.157:0/119569834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc00095f0 con 0x7f3fd0102250 2024-02-05T07:49:12.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fd4b46700 1 --2- 172.21.15.157:0/119569834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd0102250 0x7f3fd019eaa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3fc800b970 tx=0x7f3fc800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:12.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.568+0000 7f3fcd7fa700 1 -- 172.21.15.157:0/119569834 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc800d860 con 0x7f3fd0102250 2024-02-05T07:49:12.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.569+0000 7f3fcd7fa700 1 -- 172.21.15.157:0/119569834 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f3fc800de70 con 0x7f3fd0102250 2024-02-05T07:49:12.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.569+0000 7f3fd65a9700 1 -- 172.21.15.157:0/119569834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd019f620 con 0x7f3fd0102250 2024-02-05T07:49:12.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.569+0000 7f3fcd7fa700 1 -- 172.21.15.157:0/119569834 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8017400 con 0x7f3fd0102250 2024-02-05T07:49:12.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.569+0000 7f3fd65a9700 1 -- 172.21.15.157:0/119569834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd019fbf0 con 0x7f3fd0102250 2024-02-05T07:49:12.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.569+0000 7f3fd65a9700 1 -- 172.21.15.157:0/119569834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd0108aa0 con 0x7f3fd0102250 2024-02-05T07:49:12.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.570+0000 7f3fcd7fa700 1 -- 172.21.15.157:0/119569834 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f3fc800d9c0 con 0x7f3fd0102250 2024-02-05T07:49:12.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.570+0000 7f3fcd7fa700 1 --2- 172.21.15.157:0/119569834 >> v2:172.21.15.131:6800/760045032 conn(0x7f3fb806d990 0x7f3fb806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:12.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.571+0000 7f3fcd7fa700 1 -- 172.21.15.157:0/119569834 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 3789+0+0 (secure 0 0 0) 0x7f3fc8006e40 con 0x7f3fd0102250 2024-02-05T07:49:12.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.571+0000 7f3fcffff700 1 --2- 172.21.15.157:0/119569834 >> v2:172.21.15.131:6800/760045032 conn(0x7f3fb806d990 0x7f3fb806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:12.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.571+0000 7f3fcffff700 1 --2- 172.21.15.157:0/119569834 >> v2:172.21.15.131:6800/760045032 conn(0x7f3fb806d990 0x7f3fb806fe50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000b5c0 tx=0x7f3fc0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:12.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.575+0000 7f3fcd7fa700 1 -- 172.21.15.157:0/119569834 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3fc80566c0 con 0x7f3fd0102250 2024-02-05T07:49:12.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:12.778+0000 7f3fd65a9700 1 -- 172.21.15.157:0/119569834 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fd010bf40 con 0x7f3fb806d990 2024-02-05T07:49:13.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:13 smithi157 ceph-mon[33345]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-02-05T07:49:13.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:13 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:49:13.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:13 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:49:13.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:13 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:13 smithi131 ceph-mon[41654]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-02-05T07:49:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:13 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:49:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:13 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:49:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:13 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:14.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:13 smithi131 ceph-mon[29507]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-02-05T07:49:14.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:13 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:49:14.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:13 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:49:14.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:13 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:14.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:14 smithi157 ceph-mon[33345]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:49:15.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:14 smithi131 ceph-mon[41654]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:49:15.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:14 smithi131 ceph-mon[29507]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:49:16.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:15 smithi131 ceph-mon[29507]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-05T07:49:16.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.157:0/1983140926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d15bc7ec-b99f-4130-b071-3493f4ac7b1d"}]: dispatch 2024-02-05T07:49:16.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d15bc7ec-b99f-4130-b071-3493f4ac7b1d"}]: dispatch 2024-02-05T07:49:16.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d15bc7ec-b99f-4130-b071-3493f4ac7b1d"}]': finished 2024-02-05T07:49:16.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:15 smithi131 ceph-mon[29507]: osdmap e52: 8 total, 7 up, 8 in 2024-02-05T07:49:16.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:15 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:16.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:15 smithi131 ceph-mon[41654]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-05T07:49:16.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.157:0/1983140926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d15bc7ec-b99f-4130-b071-3493f4ac7b1d"}]: dispatch 2024-02-05T07:49:16.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d15bc7ec-b99f-4130-b071-3493f4ac7b1d"}]: dispatch 2024-02-05T07:49:16.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d15bc7ec-b99f-4130-b071-3493f4ac7b1d"}]': finished 2024-02-05T07:49:16.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:15 smithi131 ceph-mon[41654]: osdmap e52: 8 total, 7 up, 8 in 2024-02-05T07:49:16.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:15 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:15 smithi157 ceph-mon[33345]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-05T07:49:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.157:0/1983140926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d15bc7ec-b99f-4130-b071-3493f4ac7b1d"}]: dispatch 2024-02-05T07:49:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d15bc7ec-b99f-4130-b071-3493f4ac7b1d"}]: dispatch 2024-02-05T07:49:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d15bc7ec-b99f-4130-b071-3493f4ac7b1d"}]': finished 2024-02-05T07:49:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:15 smithi157 ceph-mon[33345]: osdmap e52: 8 total, 7 up, 8 in 2024-02-05T07:49:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:15 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:17.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.157:0/78139275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:49:17.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.157:0/78139275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:49:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.157:0/78139275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:49:18.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:17 smithi131 ceph-mon[41654]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-02-05T07:49:18.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:17 smithi131 ceph-mon[29507]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-02-05T07:49:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:17 smithi157 ceph-mon[33345]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-02-05T07:49:19.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:19 smithi157 ceph-mon[33345]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-05T07:49:20.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:19 smithi131 ceph-mon[29507]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-05T07:49:20.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:19 smithi131 ceph-mon[41654]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-05T07:49:21.855 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:21 smithi157 ceph-mon[33345]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-02-05T07:49:22.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:21 smithi131 ceph-mon[29507]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-02-05T07:49:22.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:21 smithi131 ceph-mon[41654]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-02-05T07:49:23.025 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:22 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-05T07:49:23.026 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:22 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:22 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-05T07:49:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:22 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:22 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-05T07:49:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:22 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:24.047 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:23 smithi157 ceph-mon[33345]: Deploying daemon osd.7 on smithi157 2024-02-05T07:49:24.047 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:23 smithi157 ceph-mon[33345]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:24.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:23 smithi131 ceph-mon[29507]: Deploying daemon osd.7 on smithi157 2024-02-05T07:49:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:23 smithi131 ceph-mon[29507]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:24.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:23 smithi131 ceph-mon[41654]: Deploying daemon osd.7 on smithi157 2024-02-05T07:49:24.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:23 smithi131 ceph-mon[41654]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:25.027 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:24 smithi157 ceph-mon[33345]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:25.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:24 smithi131 ceph-mon[29507]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:25.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:24 smithi131 ceph-mon[41654]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:27.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:27 smithi157 ceph-mon[33345]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:27.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:27 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:27.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:27 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:27.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:27 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:28.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:27 smithi131 ceph-mon[41654]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:28.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:27 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:28.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:27 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:28.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:27 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:28.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:27 smithi131 ceph-mon[29507]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:28.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:27 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:28.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:27 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:28.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:27 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:29.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:29 smithi157 ceph-mon[33345]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:30.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:29 smithi131 ceph-mon[41654]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:30.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:29 smithi131 ceph-mon[29507]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:31.395 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 7 on host 'smithi157' 2024-02-05T07:49:31.395 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:31.393+0000 7f3fcd7fa700 1 -- 172.21.15.157:0/119569834 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3fd010bf40 con 0x7f3fb806d990 2024-02-05T07:49:31.396 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:31.396+0000 7f3fd65a9700 1 -- 172.21.15.157:0/119569834 >> v2:172.21.15.131:6800/760045032 conn(0x7f3fb806d990 msgr2=0x7f3fb806fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:31.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:31.396+0000 7f3fd65a9700 1 --2- 172.21.15.157:0/119569834 >> v2:172.21.15.131:6800/760045032 conn(0x7f3fb806d990 0x7f3fb806fe50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000b5c0 tx=0x7f3fc0005fb0 comp rx=0 tx=0).stop 2024-02-05T07:49:31.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:31.396+0000 7f3fd65a9700 1 -- 172.21.15.157:0/119569834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd0102250 msgr2=0x7f3fd019eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:31.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:31.396+0000 7f3fd65a9700 1 --2- 172.21.15.157:0/119569834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd0102250 0x7f3fd019eaa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3fc800b970 tx=0x7f3fc800bc80 comp rx=0 tx=0).stop 2024-02-05T07:49:31.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:31.396+0000 7f3fd65a9700 1 -- 172.21.15.157:0/119569834 shutdown_connections 2024-02-05T07:49:31.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:31.396+0000 7f3fd65a9700 1 --2- 172.21.15.157:0/119569834 >> v2:172.21.15.131:6800/760045032 conn(0x7f3fb806d990 0x7f3fb806fe50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:31.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:31.396+0000 7f3fd65a9700 1 --2- 172.21.15.157:0/119569834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fd0102250 0x7f3fd019eaa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:31.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:31.396+0000 7f3fd65a9700 1 --2- 172.21.15.157:0/119569834 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f3fd0100eb0 0x7f3fd01974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:31.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:31.396+0000 7f3fd65a9700 1 --2- 172.21.15.157:0/119569834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fd0101870 0x7f3fd0197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:31.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:31.396+0000 7f3fd65a9700 1 -- 172.21.15.157:0/119569834 >> 172.21.15.157:0/119569834 conn(0x7f3fd00fb9c0 msgr2=0x7f3fd0100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:31.397 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:31.396+0000 7f3fd65a9700 1 -- 172.21.15.157:0/119569834 shutdown_connections 2024-02-05T07:49:31.398 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:31.396+0000 7f3fd65a9700 1 -- 172.21.15.157:0/119569834 wait complete. 2024-02-05T07:49:31.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:31.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:31 smithi157 ceph-mon[33345]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:31.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:31.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:31.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:31.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:31.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:31 smithi157 ceph-mon[33345]: from='osd.7 v2:172.21.15.157:6812/1810544184' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-05T07:49:31.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:31 smithi157 ceph-mon[33345]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-05T07:49:31.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:31.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:31.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:31 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:32.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:32.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[41654]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:32.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:32.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:32.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:32.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:32.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[41654]: from='osd.7 v2:172.21.15.157:6812/1810544184' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-05T07:49:32.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[41654]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-05T07:49:32.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:32.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:32.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:32.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:32.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[29507]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:32.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:32.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:32.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:32.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:32.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[29507]: from='osd.7 v2:172.21.15.157:6812/1810544184' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-05T07:49:32.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[29507]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-05T07:49:32.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:32.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:32.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:31 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:32.028 DEBUG:teuthology.orchestra.run.smithi157:osd.7> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.7.service 2024-02-05T07:49:32.030 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-02-05T07:49:32.030 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph osd stat -f json 2024-02-05T07:49:32.283 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:49:32.917 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:32 smithi157 ceph-mon[33345]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-05T07:49:32.917 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:32 smithi157 ceph-mon[33345]: from='osd.7 v2:172.21.15.157:6812/1810544184' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:49:32.917 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:32 smithi157 ceph-mon[33345]: osdmap e53: 8 total, 7 up, 8 in 2024-02-05T07:49:32.917 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:32 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:32.917 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:32 smithi157 ceph-mon[33345]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:49:33.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:32 smithi131 ceph-mon[29507]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-05T07:49:33.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:32 smithi131 ceph-mon[29507]: from='osd.7 v2:172.21.15.157:6812/1810544184' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:49:33.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:32 smithi131 ceph-mon[29507]: osdmap e53: 8 total, 7 up, 8 in 2024-02-05T07:49:33.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:32 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:33.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:32 smithi131 ceph-mon[29507]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:49:33.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:32 smithi131 ceph-mon[41654]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-05T07:49:33.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:32 smithi131 ceph-mon[41654]: from='osd.7 v2:172.21.15.157:6812/1810544184' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:49:33.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:32 smithi131 ceph-mon[41654]: osdmap e53: 8 total, 7 up, 8 in 2024-02-05T07:49:33.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:32 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:33.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:32 smithi131 ceph-mon[41654]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-02-05T07:49:33.192 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 07:49:32 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T07:49:32.920+0000 7fdc2165c700 -1 osd.7 0 waiting for initial osdmap 2024-02-05T07:49:33.192 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 07:49:32 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T07:49:32.926+0000 7fdc18c4d700 -1 osd.7 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T07:49:33.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:33 smithi131 ceph-mon[29507]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:33.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:33 smithi131 ceph-mon[29507]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-02-05T07:49:33.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:33 smithi131 ceph-mon[29507]: osdmap e54: 8 total, 7 up, 8 in 2024-02-05T07:49:33.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:33.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:33.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:33 smithi131 ceph-mon[29507]: osdmap e55: 8 total, 7 up, 8 in 2024-02-05T07:49:33.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:33 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:33.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:33 smithi131 ceph-mon[29507]: from='osd.7 ' entity='osd.7' 2024-02-05T07:49:33.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:33 smithi131 ceph-mon[41654]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:33.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:33 smithi131 ceph-mon[41654]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-02-05T07:49:33.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:33 smithi131 ceph-mon[41654]: osdmap e54: 8 total, 7 up, 8 in 2024-02-05T07:49:33.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:33.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:33.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:33 smithi131 ceph-mon[41654]: osdmap e55: 8 total, 7 up, 8 in 2024-02-05T07:49:33.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:33 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:33.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:33 smithi131 ceph-mon[41654]: from='osd.7 ' entity='osd.7' 2024-02-05T07:49:33.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:33 smithi157 ceph-mon[33345]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:33.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:33 smithi157 ceph-mon[33345]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-02-05T07:49:33.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:33 smithi157 ceph-mon[33345]: osdmap e54: 8 total, 7 up, 8 in 2024-02-05T07:49:33.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:33.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:33.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:33 smithi157 ceph-mon[33345]: osdmap e55: 8 total, 7 up, 8 in 2024-02-05T07:49:33.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:33 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:33.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:33 smithi157 ceph-mon[33345]: from='osd.7 ' entity='osd.7' 2024-02-05T07:49:33.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.996+0000 7faf0cd93700 1 -- 172.21.15.131:0/766578730 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7faf081027f0 msgr2=0x7faf081090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:33.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.996+0000 7faf0cd93700 1 --2- 172.21.15.131:0/766578730 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7faf081027f0 0x7faf081090a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faf00009f90 tx=0x7faf0000b330 comp rx=0 tx=0).stop 2024-02-05T07:49:33.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.996+0000 7faf0cd93700 1 -- 172.21.15.131:0/766578730 shutdown_connections 2024-02-05T07:49:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.996+0000 7faf0cd93700 1 --2- 172.21.15.131:0/766578730 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7faf081027f0 0x7faf081090a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.996+0000 7faf0cd93700 1 --2- 172.21.15.131:0/766578730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf08100c50 0x7faf08101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.996+0000 7faf0cd93700 1 --2- 172.21.15.131:0/766578730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf08101e50 0x7faf081022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.996+0000 7faf0cd93700 1 -- 172.21.15.131:0/766578730 >> 172.21.15.131:0/766578730 conn(0x7faf080fc210 msgr2=0x7faf080fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.996+0000 7faf0cd93700 1 -- 172.21.15.131:0/766578730 shutdown_connections 2024-02-05T07:49:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.996+0000 7faf0cd93700 1 -- 172.21.15.131:0/766578730 wait complete. 2024-02-05T07:49:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf0cd93700 1 Processor -- start 2024-02-05T07:49:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf0cd93700 1 -- start start 2024-02-05T07:49:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf0cd93700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf08100c50 0x7faf0810b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf0cd93700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7faf08101e50 0x7faf081098e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf0cd93700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf081027f0 0x7faf08109e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf0cd93700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0810a5a0 con 0x7faf081027f0 2024-02-05T07:49:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf0cd93700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0810a710 con 0x7faf08100c50 2024-02-05T07:49:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf0cd93700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf081a1520 con 0x7faf08101e50 2024-02-05T07:49:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf06d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf081027f0 0x7faf08109e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf06d9d700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf081027f0 0x7faf08109e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38176/0 (socket says 172.21.15.131:38176) 2024-02-05T07:49:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf06d9d700 1 -- 172.21.15.131:0/2105118380 learned_addr learned my addr 172.21.15.131:0/2105118380 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:49:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf0659c700 1 --2- 172.21.15.131:0/2105118380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf08100c50 0x7faf0810b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf05d9b700 1 --2- 172.21.15.131:0/2105118380 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7faf08101e50 0x7faf081098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf06d9d700 1 -- 172.21.15.131:0/2105118380 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7faf08101e50 msgr2=0x7faf081098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf06d9d700 1 --2- 172.21.15.131:0/2105118380 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7faf08101e50 0x7faf081098e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf06d9d700 1 -- 172.21.15.131:0/2105118380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf08100c50 msgr2=0x7faf0810b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf06d9d700 1 --2- 172.21.15.131:0/2105118380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf08100c50 0x7faf0810b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf06d9d700 1 -- 172.21.15.131:0/2105118380 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc0095f0 con 0x7faf081027f0 2024-02-05T07:49:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf0659c700 1 --2- 172.21.15.131:0/2105118380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf08100c50 0x7faf0810b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:49:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faf06d9d700 1 --2- 172.21.15.131:0/2105118380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf081027f0 0x7faf08109e20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf00009f60 tx=0x7faf00004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:34.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.997+0000 7faef77fe700 1 -- 172.21.15.131:0/2105118380 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0000e040 con 0x7faf081027f0 2024-02-05T07:49:34.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.998+0000 7faef77fe700 1 -- 172.21.15.131:0/2105118380 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7faf00008ed0 con 0x7faf081027f0 2024-02-05T07:49:34.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.998+0000 7faef77fe700 1 -- 172.21.15.131:0/2105118380 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf000052a0 con 0x7faf081027f0 2024-02-05T07:49:34.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.998+0000 7faf0cd93700 1 -- 172.21.15.131:0/2105118380 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf00009c40 con 0x7faf081027f0 2024-02-05T07:49:34.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.998+0000 7faf0cd93700 1 -- 172.21.15.131:0/2105118380 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf081a1ad0 con 0x7faf081027f0 2024-02-05T07:49:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.998+0000 7faf0cd93700 1 -- 172.21.15.131:0/2105118380 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf08107f40 con 0x7faf081027f0 2024-02-05T07:49:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.999+0000 7faef77fe700 1 -- 172.21.15.131:0/2105118380 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7faf0001d470 con 0x7faf081027f0 2024-02-05T07:49:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:33.999+0000 7faef77fe700 1 --2- 172.21.15.131:0/2105118380 >> v2:172.21.15.131:6800/760045032 conn(0x7faef006d990 0x7faef006fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.000+0000 7faef77fe700 1 -- 172.21.15.131:0/2105118380 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4114+0+0 (secure 0 0 0) 0x7faf0008de00 con 0x7faf081027f0 2024-02-05T07:49:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.000+0000 7faf0659c700 1 --2- 172.21.15.131:0/2105118380 >> v2:172.21.15.131:6800/760045032 conn(0x7faef006d990 0x7faef006fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.000+0000 7faf0659c700 1 --2- 172.21.15.131:0/2105118380 >> v2:172.21.15.131:6800/760045032 conn(0x7faef006d990 0x7faef006fe50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf08101cb0 tx=0x7faef8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.004+0000 7faef77fe700 1 -- 172.21.15.131:0/2105118380 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faf00057fd0 con 0x7faf081027f0 2024-02-05T07:49:34.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.196+0000 7faf0cd93700 1 -- 172.21.15.131:0/2105118380 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7faf0810ce40 con 0x7faf081027f0 2024-02-05T07:49:34.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.197+0000 7faef77fe700 1 -- 172.21.15.131:0/2105118380 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v56) v1 ==== 74+0+130 (secure 0 0 0) 0x7faf0005b690 con 0x7faf081027f0 2024-02-05T07:49:34.198 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:49:34.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.199+0000 7faf0cd93700 1 -- 172.21.15.131:0/2105118380 >> v2:172.21.15.131:6800/760045032 conn(0x7faef006d990 msgr2=0x7faef006fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:34.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.199+0000 7faf0cd93700 1 --2- 172.21.15.131:0/2105118380 >> v2:172.21.15.131:6800/760045032 conn(0x7faef006d990 0x7faef006fe50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf08101cb0 tx=0x7faef8009260 comp rx=0 tx=0).stop 2024-02-05T07:49:34.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.199+0000 7faf0cd93700 1 -- 172.21.15.131:0/2105118380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf081027f0 msgr2=0x7faf08109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:34.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.199+0000 7faf0cd93700 1 --2- 172.21.15.131:0/2105118380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf081027f0 0x7faf08109e20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf00009f60 tx=0x7faf00004c80 comp rx=0 tx=0).stop 2024-02-05T07:49:34.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.199+0000 7faf0cd93700 1 -- 172.21.15.131:0/2105118380 shutdown_connections 2024-02-05T07:49:34.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.199+0000 7faf0cd93700 1 --2- 172.21.15.131:0/2105118380 >> v2:172.21.15.131:6800/760045032 conn(0x7faef006d990 0x7faef006fe50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:34.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.199+0000 7faf0cd93700 1 --2- 172.21.15.131:0/2105118380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf081027f0 0x7faf08109e20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:34.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.199+0000 7faf0cd93700 1 --2- 172.21.15.131:0/2105118380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf08100c50 0x7faf0810b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:34.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.199+0000 7faf0cd93700 1 --2- 172.21.15.131:0/2105118380 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7faf08101e50 0x7faf081098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:34.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.199+0000 7faf0cd93700 1 -- 172.21.15.131:0/2105118380 >> 172.21.15.131:0/2105118380 conn(0x7faf080fc210 msgr2=0x7faf08105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:34.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.199+0000 7faf0cd93700 1 -- 172.21.15.131:0/2105118380 shutdown_connections 2024-02-05T07:49:34.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:34.199+0000 7faf0cd93700 1 -- 172.21.15.131:0/2105118380 wait complete. 2024-02-05T07:49:34.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:34 smithi131 ceph-mon[41654]: purged_snaps scrub starts 2024-02-05T07:49:34.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:34 smithi131 ceph-mon[41654]: purged_snaps scrub ok 2024-02-05T07:49:34.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:34 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:34.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:34 smithi131 ceph-mon[41654]: osd.7 v2:172.21.15.157:6812/1810544184 boot 2024-02-05T07:49:34.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:34 smithi131 ceph-mon[41654]: osdmap e56: 8 total, 8 up, 8 in 2024-02-05T07:49:34.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:34 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:34.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2105118380' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-05T07:49:34.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:34 smithi131 ceph-mon[29507]: purged_snaps scrub starts 2024-02-05T07:49:34.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:34 smithi131 ceph-mon[29507]: purged_snaps scrub ok 2024-02-05T07:49:34.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:34 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:34.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:34 smithi131 ceph-mon[29507]: osd.7 v2:172.21.15.157:6812/1810544184 boot 2024-02-05T07:49:34.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:34 smithi131 ceph-mon[29507]: osdmap e56: 8 total, 8 up, 8 in 2024-02-05T07:49:34.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:34 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:34.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2105118380' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-05T07:49:34.884 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1707119373,"num_in_osds":8,"osd_in_since":1707119355,"num_remapped_pgs":1} 2024-02-05T07:49:34.885 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-05T07:49:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:34 smithi157 ceph-mon[33345]: purged_snaps scrub starts 2024-02-05T07:49:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:34 smithi157 ceph-mon[33345]: purged_snaps scrub ok 2024-02-05T07:49:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:34 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:34 smithi157 ceph-mon[33345]: osd.7 v2:172.21.15.157:6812/1810544184 boot 2024-02-05T07:49:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:34 smithi157 ceph-mon[33345]: osdmap e56: 8 total, 8 up, 8 in 2024-02-05T07:49:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:34 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:49:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2105118380' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-05T07:49:35.130 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:49:35.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[29507]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:35.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[29507]: osdmap e57: 8 total, 8 up, 8 in 2024-02-05T07:49:35.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:35.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:35.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:35.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:35.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:35.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:35.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:35.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:35.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:35.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:35.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[41654]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:35.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[41654]: osdmap e57: 8 total, 8 up, 8 in 2024-02-05T07:49:35.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:35.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:35.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:35.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:35.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:35.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:35.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:35.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:35.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:35.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:35 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:35 smithi157 ceph-mon[33345]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:49:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:35 smithi157 ceph-mon[33345]: osdmap e57: 8 total, 8 up, 8 in 2024-02-05T07:49:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:35 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:35 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:35 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:35 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:36.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:35 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:36.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:35 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:49:36.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:35 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:36.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:35 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:36.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:35 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:36.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:35 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:36.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.860+0000 7f5b2d721700 1 -- 172.21.15.131:0/2295871586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b28107e40 msgr2=0x7f5b28108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:36.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.860+0000 7f5b2d721700 1 --2- 172.21.15.131:0/2295871586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b28107e40 0x7f5b28108220 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5b180098f0 tx=0x7f5b18009c00 comp rx=0 tx=0).stop 2024-02-05T07:49:36.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.860+0000 7f5b2d721700 1 -- 172.21.15.131:0/2295871586 shutdown_connections 2024-02-05T07:49:36.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.860+0000 7f5b2d721700 1 --2- 172.21.15.131:0/2295871586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b28107e40 0x7f5b28108220 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:36.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.860+0000 7f5b2d721700 1 --2- 172.21.15.131:0/2295871586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b28102790 0x7f5b2810eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:36.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.860+0000 7f5b2d721700 1 --2- 172.21.15.131:0/2295871586 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f5b28101dd0 0x7f5b28102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:36.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.860+0000 7f5b2d721700 1 -- 172.21.15.131:0/2295871586 >> 172.21.15.131:0/2295871586 conn(0x7f5b28078df0 msgr2=0x7f5b28079200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:36.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.860+0000 7f5b2d721700 1 -- 172.21.15.131:0/2295871586 shutdown_connections 2024-02-05T07:49:36.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.860+0000 7f5b2d721700 1 -- 172.21.15.131:0/2295871586 wait complete. 2024-02-05T07:49:36.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.861+0000 7f5b2d721700 1 Processor -- start 2024-02-05T07:49:36.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.861+0000 7f5b2d721700 1 -- start start 2024-02-05T07:49:36.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.861+0000 7f5b2d721700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f5b28101dd0 0x7f5b2819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:36.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.861+0000 7f5b2d721700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b28102790 0x7f5b2819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:36.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.861+0000 7f5b2d721700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b28107e40 0x7f5b281977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:36.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.861+0000 7f5b2d721700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b28197e80 con 0x7f5b28107e40 2024-02-05T07:49:36.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.861+0000 7f5b2d721700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b28197ff0 con 0x7f5b28102790 2024-02-05T07:49:36.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.861+0000 7f5b2d721700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b281981b0 con 0x7f5b28101dd0 2024-02-05T07:49:36.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.861+0000 7f5b26ffd700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f5b28101dd0 0x7f5b2819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:36.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.862+0000 7f5b26ffd700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f5b28101dd0 0x7f5b2819d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:40050/0 (socket says 172.21.15.131:40050) 2024-02-05T07:49:36.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.862+0000 7f5b26ffd700 1 -- 172.21.15.131:0/1936812751 learned_addr learned my addr 172.21.15.131:0/1936812751 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:49:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.862+0000 7f5b277fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b28107e40 0x7f5b281977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.862+0000 7f5b267fc700 1 --2- 172.21.15.131:0/1936812751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b28102790 0x7f5b2819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.862+0000 7f5b26ffd700 1 -- 172.21.15.131:0/1936812751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b28102790 msgr2=0x7f5b2819db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.862+0000 7f5b26ffd700 1 --2- 172.21.15.131:0/1936812751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b28102790 0x7f5b2819db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.862+0000 7f5b26ffd700 1 -- 172.21.15.131:0/1936812751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b28107e40 msgr2=0x7f5b281977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.862+0000 7f5b26ffd700 1 --2- 172.21.15.131:0/1936812751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b28107e40 0x7f5b281977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.862+0000 7f5b26ffd700 1 -- 172.21.15.131:0/1936812751 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c0095f0 con 0x7f5b28101dd0 2024-02-05T07:49:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.862+0000 7f5b277fe700 1 --2- 172.21.15.131:0/1936812751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b28107e40 0x7f5b281977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:49:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.862+0000 7f5b26ffd700 1 --2- 172.21.15.131:0/1936812751 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f5b28101dd0 0x7f5b2819d650 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800f9e0 tx=0x7f5b1800fa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.862+0000 7f5b0ffff700 1 -- 172.21.15.131:0/1936812751 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1801c070 con 0x7f5b28101dd0 2024-02-05T07:49:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.862+0000 7f5b2d721700 1 -- 172.21.15.131:0/1936812751 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b18009520 con 0x7f5b28101dd0 2024-02-05T07:49:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.862+0000 7f5b0ffff700 1 -- 172.21.15.131:0/1936812751 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f5b1800ba90 con 0x7f5b28101dd0 2024-02-05T07:49:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.862+0000 7f5b0ffff700 1 -- 172.21.15.131:0/1936812751 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18004930 con 0x7f5b28101dd0 2024-02-05T07:49:36.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.862+0000 7f5b2d721700 1 -- 172.21.15.131:0/1936812751 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b281a44b0 con 0x7f5b28101dd0 2024-02-05T07:49:36.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.863+0000 7f5b2d721700 1 -- 172.21.15.131:0/1936812751 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b28103460 con 0x7f5b28101dd0 2024-02-05T07:49:36.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.864+0000 7f5b0ffff700 1 -- 172.21.15.131:0/1936812751 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f5b18020070 con 0x7f5b28101dd0 2024-02-05T07:49:36.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.864+0000 7f5b0ffff700 1 --2- 172.21.15.131:0/1936812751 >> v2:172.21.15.131:6800/760045032 conn(0x7f5b1006d990 0x7f5b1006fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:36.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.864+0000 7f5b0ffff700 1 -- 172.21.15.131:0/1936812751 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4081+0+0 (secure 0 0 0) 0x7f5b1808d680 con 0x7f5b28101dd0 2024-02-05T07:49:36.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.864+0000 7f5b267fc700 1 --2- 172.21.15.131:0/1936812751 >> v2:172.21.15.131:6800/760045032 conn(0x7f5b1006d990 0x7f5b1006fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:36.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.864+0000 7f5b267fc700 1 --2- 172.21.15.131:0/1936812751 >> v2:172.21.15.131:6800/760045032 conn(0x7f5b1006d990 0x7f5b1006fe50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5b28112160 tx=0x7f5b1c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:36.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:36.869+0000 7f5b0ffff700 1 -- 172.21.15.131:0/1936812751 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b180577f0 con 0x7f5b28101dd0 2024-02-05T07:49:37.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:36 smithi131 ceph-mon[29507]: Detected new or changed devices on smithi157 2024-02-05T07:49:37.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:36 smithi131 ceph-mon[29507]: Adjusting osd_memory_target on smithi157 to 4272M 2024-02-05T07:49:37.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:36 smithi131 ceph-mon[29507]: osdmap e58: 8 total, 8 up, 8 in 2024-02-05T07:49:37.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:36 smithi131 ceph-mon[41654]: Detected new or changed devices on smithi157 2024-02-05T07:49:37.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:36 smithi131 ceph-mon[41654]: Adjusting osd_memory_target on smithi157 to 4272M 2024-02-05T07:49:37.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:36 smithi131 ceph-mon[41654]: osdmap e58: 8 total, 8 up, 8 in 2024-02-05T07:49:37.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:37.064+0000 7f5b2d721700 1 -- 172.21.15.131:0/1936812751 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b280689e0 con 0x7f5b28101dd0 2024-02-05T07:49:37.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:37.065+0000 7f5b0ffff700 1 -- 172.21.15.131:0/1936812751 <== mon.2 v2:172.21.15.131:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+11544 (secure 0 0 0) 0x7f5b1805aeb0 con 0x7f5b28101dd0 2024-02-05T07:49:37.066 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:49:37.066 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":59,"fsid":"4018fbe8-c3fa-11ee-95b6-87774f69a715","created":"2024-02-05T07:44:07.660911+0000","modified":"2024-02-05T07:49:36.727198+0000","last_up_change":"2024-02-05T07:49:33.716478+0000","last_in_change":"2024-02-05T07:49:15.544169+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-05T07:47:28.930141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f89dd964-ec08-4f4b-9e8f-91bea0156d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6801","nonce":1614402360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1614402360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1614402360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6803","nonce":1614402360}]},"public_addr":"172.21.15.131:6801/1614402360","cluster_addr":"172.21.15.131:6802/1614402360","heartbeat_back_addr":"172.21.15.131:6804/1614402360","heartbeat_front_addr":"172.21.15.131:6803/1614402360","state":["exists","up"]},{"osd":1,"uuid":"b6068c48-3bc2-4728-9924-66bafeff3703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6805","nonce":1256644882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1256644882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1256644882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6807","nonce":1256644882}]},"public_addr":"172.21.15.131:6805/1256644882","cluster_addr":"172.21.15.131:6806/1256644882","heartbeat_back_addr":"172.21.15.131:6808/1256644882","heartbeat_front_addr":"172.21.15.131:6807/1256644882","state":["exists","up"]},{"osd":2,"uuid":"8d0cfb98-091c-467e-abe5-abe3b8542323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6809","nonce":1463881636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1463881636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1463881636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6811","nonce":1463881636}]},"public_addr":"172.21.15.131:6809/1463881636","cluster_addr":"172.21.15.131:6810/1463881636","heartbeat_back_addr":"172.21.15.131:6812/1463881636","heartbeat_front_addr":"172.21.15.131:6811/1463881636","state":["exists","up"]},{"osd":3,"uuid":"3e262dfc-6d5a-4f3f-b605-6c92e6839f62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":3290626285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3290626285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3290626285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6815","nonce":3290626285}]},"public_addr":"172.21.15.131:6813/3290626285","cluster_addr":"172.21.15.131:6814/3290626285","heartbeat_back_addr":"172.21.15.131:6816/3290626285","heartbeat_front_addr":"172.21.15.131:6815/3290626285","state":["exists","up"]},{"osd":4,"uuid":"f6925a13-9a72-42bd-97b2-970f9e709dad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1828659015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":1828659015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6803","nonce":1828659015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1828659015}]},"public_addr":"172.21.15.157:6800/1828659015","cluster_addr":"172.21.15.157:6801/1828659015","heartbeat_back_addr":"172.21.15.157:6803/1828659015","heartbeat_front_addr":"172.21.15.157:6802/1828659015","state":["exists","up"]},{"osd":5,"uuid":"96c3644d-9b3a-4826-befe-0f6cc4af9baf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3589104149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":3589104149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":3589104149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3589104149}]},"public_addr":"172.21.15.157:6804/3589104149","cluster_addr":"172.21.15.157:6805/3589104149","heartbeat_back_addr":"172.21.15.157:6807/3589104149","heartbeat_front_addr":"172.21.15.157:6806/3589104149","state":["exists","up"]},{"osd":6,"uuid":"24119b57-55ed-4b51-b25e-7a6cea556af3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":204226792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":204226792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":204226792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":204226792}]},"public_addr":"172.21.15.157:6808/204226792","cluster_addr":"172.21.15.157:6809/204226792","heartbeat_back_addr":"172.21.15.157:6811/204226792","heartbeat_front_addr":"172.21.15.157:6810/204226792","state":["exists","up"]},{"osd":7,"uuid":"d15bc7ec-b99f-4130-b071-3493f4ac7b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1810544184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":1810544184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":1810544184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1810544184}]},"public_addr":"172.21.15.157:6812/1810544184","cluster_addr":"172.21.15.157:6813/1810544184","heartbeat_back_addr":"172.21.15.157:6815/1810544184","heartbeat_front_addr":"172.21.15.157:6814/1810544184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:46:34.040947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:46:59.284998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:47:25.195950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:47:51.338667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:48:14.496069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:48:39.323248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:49:05.597163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:49:31.741281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/1270355705":"2024-02-06T07:45:10.598786+0000","172.21.15.131:6800/2947584178":"2024-02-06T07:45:10.598786+0000","172.21.15.131:0/3554035799":"2024-02-06T07:44:42.598552+0000","172.21.15.131:0/3462494148":"2024-02-06T07:44:42.598552+0000","172.21.15.131:6800/1416975616":"2024-02-06T07:44:42.598552+0000","172.21.15.131:0/219975829":"2024-02-06T07:44:42.598552+0000","172.21.15.131:0/1166974614":"2024-02-06T07:45:10.598786+0000","172.21.15.131:0/3288446474":"2024-02-06T07:45:10.598786+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-05T07:49:37.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:37.067+0000 7f5b2d721700 1 -- 172.21.15.131:0/1936812751 >> v2:172.21.15.131:6800/760045032 conn(0x7f5b1006d990 msgr2=0x7f5b1006fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:37.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:37.067+0000 7f5b2d721700 1 --2- 172.21.15.131:0/1936812751 >> v2:172.21.15.131:6800/760045032 conn(0x7f5b1006d990 0x7f5b1006fe50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5b28112160 tx=0x7f5b1c009310 comp rx=0 tx=0).stop 2024-02-05T07:49:37.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:37.067+0000 7f5b2d721700 1 -- 172.21.15.131:0/1936812751 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f5b28101dd0 msgr2=0x7f5b2819d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:37.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:37.067+0000 7f5b2d721700 1 --2- 172.21.15.131:0/1936812751 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f5b28101dd0 0x7f5b2819d650 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800f9e0 tx=0x7f5b1800fa10 comp rx=0 tx=0).stop 2024-02-05T07:49:37.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:37.067+0000 7f5b2d721700 1 -- 172.21.15.131:0/1936812751 shutdown_connections 2024-02-05T07:49:37.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:37.067+0000 7f5b2d721700 1 --2- 172.21.15.131:0/1936812751 >> v2:172.21.15.131:6800/760045032 conn(0x7f5b1006d990 0x7f5b1006fe50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:37.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:37.067+0000 7f5b2d721700 1 --2- 172.21.15.131:0/1936812751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5b28107e40 0x7f5b281977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:37.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:37.067+0000 7f5b2d721700 1 --2- 172.21.15.131:0/1936812751 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f5b28101dd0 0x7f5b2819d650 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:37.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:37.067+0000 7f5b2d721700 1 --2- 172.21.15.131:0/1936812751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b28102790 0x7f5b2819db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:37.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:37.067+0000 7f5b2d721700 1 -- 172.21.15.131:0/1936812751 >> 172.21.15.131:0/1936812751 conn(0x7f5b28078df0 msgr2=0x7f5b28105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:37.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:37.067+0000 7f5b2d721700 1 -- 172.21.15.131:0/1936812751 shutdown_connections 2024-02-05T07:49:37.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:37.067+0000 7f5b2d721700 1 -- 172.21.15.131:0/1936812751 wait complete. 2024-02-05T07:49:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:36 smithi157 ceph-mon[33345]: Detected new or changed devices on smithi157 2024-02-05T07:49:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:36 smithi157 ceph-mon[33345]: Adjusting osd_memory_target on smithi157 to 4272M 2024-02-05T07:49:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:36 smithi157 ceph-mon[33345]: osdmap e58: 8 total, 8 up, 8 in 2024-02-05T07:49:37.796 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-05T07:47:28.930141+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-05T07:49:37.796 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-05T07:49:38.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:37 smithi131 ceph-mon[29507]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:49:38.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:37 smithi131 ceph-mon[29507]: osdmap e59: 8 total, 8 up, 8 in 2024-02-05T07:49:38.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1936812751' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:49:38.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:37 smithi131 ceph-mon[41654]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:49:38.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:37 smithi131 ceph-mon[41654]: osdmap e59: 8 total, 8 up, 8 in 2024-02-05T07:49:38.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1936812751' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:49:38.032 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:49:38.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:37 smithi157 ceph-mon[33345]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:49:38.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:37 smithi157 ceph-mon[33345]: osdmap e59: 8 total, 8 up, 8 in 2024-02-05T07:49:38.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1936812751' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:49:39.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:38 smithi131 ceph-mon[29507]: osdmap e60: 8 total, 8 up, 8 in 2024-02-05T07:49:39.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:38 smithi131 ceph-mon[41654]: osdmap e60: 8 total, 8 up, 8 in 2024-02-05T07:49:39.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:38 smithi157 ceph-mon[33345]: osdmap e60: 8 total, 8 up, 8 in 2024-02-05T07:49:39.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.579+0000 7f46278a4700 1 -- 172.21.15.131:0/2273958971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46201029b0 msgr2=0x7f4620106950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:39.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.579+0000 7f46278a4700 1 --2- 172.21.15.131:0/2273958971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46201029b0 0x7f4620106950 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f46100098f0 tx=0x7f4610009c00 comp rx=0 tx=0).stop 2024-02-05T07:49:39.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.579+0000 7f46278a4700 1 -- 172.21.15.131:0/2273958971 shutdown_connections 2024-02-05T07:49:39.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.579+0000 7f46278a4700 1 --2- 172.21.15.131:0/2273958971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4620107030 0x7f462010ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:39.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.579+0000 7f46278a4700 1 --2- 172.21.15.131:0/2273958971 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f4620102000 0x7f46201023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:39.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.579+0000 7f46278a4700 1 --2- 172.21.15.131:0/2273958971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46201029b0 0x7f4620106950 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:39.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.579+0000 7f46278a4700 1 -- 172.21.15.131:0/2273958971 >> 172.21.15.131:0/2273958971 conn(0x7f4620078db0 msgr2=0x7f46200791c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:39.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.579+0000 7f46278a4700 1 -- 172.21.15.131:0/2273958971 shutdown_connections 2024-02-05T07:49:39.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.579+0000 7f46278a4700 1 -- 172.21.15.131:0/2273958971 wait complete. 2024-02-05T07:49:39.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.579+0000 7f46278a4700 1 Processor -- start 2024-02-05T07:49:39.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.579+0000 7f46278a4700 1 -- start start 2024-02-05T07:49:39.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f46278a4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4620102000 0x7f4620074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:39.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f46278a4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46201029b0 0x7f4620074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:39.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f46278a4700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f4620107030 0x7f4620074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:39.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f46278a4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a5db0 con 0x7f46201029b0 2024-02-05T07:49:39.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f46278a4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a5f20 con 0x7f4620102000 2024-02-05T07:49:39.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f46278a4700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a60e0 con 0x7f4620107030 2024-02-05T07:49:39.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f4624e3f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46201029b0 0x7f4620074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:39.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f4625e41700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f4620107030 0x7f4620074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:39.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f4624e3f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46201029b0 0x7f4620074560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59866/0 (socket says 172.21.15.131:59866) 2024-02-05T07:49:39.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f4625e41700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f4620107030 0x7f4620074aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:50546/0 (socket says 172.21.15.131:50546) 2024-02-05T07:49:39.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f4624e3f700 1 -- 172.21.15.131:0/874260543 learned_addr learned my addr 172.21.15.131:0/874260543 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:49:39.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f4625640700 1 --2- 172.21.15.131:0/874260543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4620102000 0x7f4620074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:39.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f4624e3f700 1 -- 172.21.15.131:0/874260543 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f4620107030 msgr2=0x7f4620074aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:39.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f4624e3f700 1 --2- 172.21.15.131:0/874260543 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f4620107030 0x7f4620074aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:39.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f4624e3f700 1 -- 172.21.15.131:0/874260543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4620102000 msgr2=0x7f4620074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:39.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f4624e3f700 1 --2- 172.21.15.131:0/874260543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4620102000 0x7f4620074020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:39.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f4624e3f700 1 -- 172.21.15.131:0/874260543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c0095f0 con 0x7f46201029b0 2024-02-05T07:49:39.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f4625640700 1 --2- 172.21.15.131:0/874260543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4620102000 0x7f4620074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T07:49:39.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f4624e3f700 1 --2- 172.21.15.131:0/874260543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46201029b0 0x7f4620074560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4610009230 tx=0x7f461000f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:39.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f46167fc700 1 -- 172.21.15.131:0/874260543 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461001c070 con 0x7f46201029b0 2024-02-05T07:49:39.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f46167fc700 1 -- 172.21.15.131:0/874260543 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f461000bb10 con 0x7f46201029b0 2024-02-05T07:49:39.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f46278a4700 1 -- 172.21.15.131:0/874260543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4610009520 con 0x7f46201029b0 2024-02-05T07:49:39.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.580+0000 7f46167fc700 1 -- 172.21.15.131:0/874260543 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100038e0 con 0x7f46201029b0 2024-02-05T07:49:39.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.581+0000 7f46278a4700 1 -- 172.21.15.131:0/874260543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46201a65e0 con 0x7f46201029b0 2024-02-05T07:49:39.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.581+0000 7f46278a4700 1 -- 172.21.15.131:0/874260543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46200689e0 con 0x7f46201029b0 2024-02-05T07:49:39.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.582+0000 7f46167fc700 1 -- 172.21.15.131:0/874260543 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f4610020030 con 0x7f46201029b0 2024-02-05T07:49:39.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.582+0000 7f46167fc700 1 --2- 172.21.15.131:0/874260543 >> v2:172.21.15.131:6800/760045032 conn(0x7f460c06d990 0x7f460c06fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:39.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.582+0000 7f46167fc700 1 -- 172.21.15.131:0/874260543 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4081+0+0 (secure 0 0 0) 0x7f461008da10 con 0x7f46201029b0 2024-02-05T07:49:39.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.582+0000 7f4625640700 1 --2- 172.21.15.131:0/874260543 >> v2:172.21.15.131:6800/760045032 conn(0x7f460c06d990 0x7f460c06fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:39.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.583+0000 7f4625640700 1 --2- 172.21.15.131:0/874260543 >> v2:172.21.15.131:6800/760045032 conn(0x7f460c06d990 0x7f460c06fe50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f461c005fd0 tx=0x7f461c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:39.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.587+0000 7f46167fc700 1 -- 172.21.15.131:0/874260543 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4610057b80 con 0x7f46201029b0 2024-02-05T07:49:39.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.780+0000 7f46278a4700 1 -- 172.21.15.131:0/874260543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4620111460 con 0x7f46201029b0 2024-02-05T07:49:39.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.780+0000 7f46167fc700 1 -- 172.21.15.131:0/874260543 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f461005b240 con 0x7f46201029b0 2024-02-05T07:49:39.781 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-02-05T07:49:39.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.782+0000 7f46278a4700 1 -- 172.21.15.131:0/874260543 >> v2:172.21.15.131:6800/760045032 conn(0x7f460c06d990 msgr2=0x7f460c06fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:39.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.782+0000 7f46278a4700 1 --2- 172.21.15.131:0/874260543 >> v2:172.21.15.131:6800/760045032 conn(0x7f460c06d990 0x7f460c06fe50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f461c005fd0 tx=0x7f461c009310 comp rx=0 tx=0).stop 2024-02-05T07:49:39.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.782+0000 7f46278a4700 1 -- 172.21.15.131:0/874260543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46201029b0 msgr2=0x7f4620074560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:39.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.782+0000 7f46278a4700 1 --2- 172.21.15.131:0/874260543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46201029b0 0x7f4620074560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4610009230 tx=0x7f461000f9e0 comp rx=0 tx=0).stop 2024-02-05T07:49:39.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.782+0000 7f46278a4700 1 -- 172.21.15.131:0/874260543 shutdown_connections 2024-02-05T07:49:39.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.782+0000 7f46278a4700 1 --2- 172.21.15.131:0/874260543 >> v2:172.21.15.131:6800/760045032 conn(0x7f460c06d990 0x7f460c06fe50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:39.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.782+0000 7f46278a4700 1 --2- 172.21.15.131:0/874260543 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f4620107030 0x7f4620074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:39.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.782+0000 7f46278a4700 1 --2- 172.21.15.131:0/874260543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4620102000 0x7f4620074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:39.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.782+0000 7f46278a4700 1 --2- 172.21.15.131:0/874260543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46201029b0 0x7f4620074560 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:39.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.782+0000 7f46278a4700 1 -- 172.21.15.131:0/874260543 >> 172.21.15.131:0/874260543 conn(0x7f4620078db0 msgr2=0x7f46200ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:39.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.783+0000 7f46278a4700 1 -- 172.21.15.131:0/874260543 shutdown_connections 2024-02-05T07:49:39.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:49:39.783+0000 7f46278a4700 1 -- 172.21.15.131:0/874260543 wait complete. 2024-02-05T07:49:40.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:39 smithi131 ceph-mon[41654]: pgmap v184: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-05T07:49:40.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:39 smithi131 ceph-mon[29507]: pgmap v184: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-05T07:49:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:39 smithi157 ceph-mon[33345]: pgmap v184: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-05T07:49:40.435 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi131 2024-02-05T07:49:40.435 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi131=foo.a' 2024-02-05T07:49:40.685 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:49:41.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:40 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/874260543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-05T07:49:41.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:40 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/874260543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-05T07:49:41.026 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:40 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/874260543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-05T07:49:41.956 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:41 smithi157 ceph-mon[33345]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-05T07:49:42.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:41 smithi131 ceph-mon[29507]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-05T07:49:42.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:41 smithi131 ceph-mon[41654]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-05T07:49:42.192 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.189+0000 7f5fe619a700 1 -- 172.21.15.157:0/2565230491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe0107aa0 msgr2=0x7f5fe0109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:42.192 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.189+0000 7f5fe619a700 1 --2- 172.21.15.157:0/2565230491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe0107aa0 0x7f5fe0109eb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800b0a0 tx=0x7f5fd800b3b0 comp rx=0 tx=0).stop 2024-02-05T07:49:42.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.189+0000 7f5fe619a700 1 -- 172.21.15.157:0/2565230491 shutdown_connections 2024-02-05T07:49:42.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.189+0000 7f5fe619a700 1 --2- 172.21.15.157:0/2565230491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fe0102840 0x7f5fe0104c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:42.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.189+0000 7f5fe619a700 1 --2- 172.21.15.157:0/2565230491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe0107aa0 0x7f5fe0109eb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:42.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.189+0000 7f5fe619a700 1 --2- 172.21.15.157:0/2565230491 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f5fe0105170 0x7f5fe0107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:42.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.189+0000 7f5fe619a700 1 -- 172.21.15.157:0/2565230491 >> 172.21.15.157:0/2565230491 conn(0x7f5fe00fc1f0 msgr2=0x7f5fe00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:42.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.189+0000 7f5fe619a700 1 -- 172.21.15.157:0/2565230491 shutdown_connections 2024-02-05T07:49:42.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.189+0000 7f5fe619a700 1 -- 172.21.15.157:0/2565230491 wait complete. 2024-02-05T07:49:42.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.190+0000 7f5fe619a700 1 Processor -- start 2024-02-05T07:49:42.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.190+0000 7f5fe619a700 1 -- start start 2024-02-05T07:49:42.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.190+0000 7f5fe619a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fe0102840 0x7f5fe01021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:42.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.190+0000 7f5fe619a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe0105170 0x7f5fe01007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:42.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.190+0000 7f5fe619a700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f5fe0107aa0 0x7f5fe0100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:42.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.190+0000 7f5fe619a700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe01014b0 con 0x7f5fe0102840 2024-02-05T07:49:42.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.190+0000 7f5fe619a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe0101620 con 0x7f5fe0105170 2024-02-05T07:49:42.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.190+0000 7f5fe619a700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe01a1600 con 0x7f5fe0107aa0 2024-02-05T07:49:42.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.190+0000 7f5fdeffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe0105170 0x7f5fe01007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:42.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.190+0000 7f5fdeffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe0105170 0x7f5fe01007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43098/0 (socket says 172.21.15.157:43098) 2024-02-05T07:49:42.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.190+0000 7f5fdeffd700 1 -- 172.21.15.157:0/3343183426 learned_addr learned my addr 172.21.15.157:0/3343183426 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:49:42.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.190+0000 7f5fdffff700 1 --2- 172.21.15.157:0/3343183426 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f5fe0107aa0 0x7f5fe0100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:42.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.190+0000 7f5fdf7fe700 1 --2- 172.21.15.157:0/3343183426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fe0102840 0x7f5fe01021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:42.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.191+0000 7f5fdeffd700 1 -- 172.21.15.157:0/3343183426 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f5fe0107aa0 msgr2=0x7f5fe0100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:42.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.191+0000 7f5fdeffd700 1 --2- 172.21.15.157:0/3343183426 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f5fe0107aa0 0x7f5fe0100d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:42.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.191+0000 7f5fdeffd700 1 -- 172.21.15.157:0/3343183426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fe0102840 msgr2=0x7f5fe01021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:42.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.191+0000 7f5fdeffd700 1 --2- 172.21.15.157:0/3343183426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fe0102840 0x7f5fe01021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:42.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.191+0000 7f5fdeffd700 1 -- 172.21.15.157:0/3343183426 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd8009d00 con 0x7f5fe0105170 2024-02-05T07:49:42.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.191+0000 7f5fdf7fe700 1 --2- 172.21.15.157:0/3343183426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fe0102840 0x7f5fe01021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-05T07:49:42.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.191+0000 7f5fdeffd700 1 --2- 172.21.15.157:0/3343183426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe0105170 0x7f5fe01007f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400c920 tx=0x7f5fd400cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:42.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.191+0000 7f5fdcff9700 1 -- 172.21.15.157:0/3343183426 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4007b00 con 0x7f5fe0105170 2024-02-05T07:49:42.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.191+0000 7f5fdcff9700 1 -- 172.21.15.157:0/3343183426 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f5fd4004ce0 con 0x7f5fe0105170 2024-02-05T07:49:42.195 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.191+0000 7f5fdcff9700 1 -- 172.21.15.157:0/3343183426 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4005550 con 0x7f5fe0105170 2024-02-05T07:49:42.195 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.191+0000 7f5fe619a700 1 -- 172.21.15.157:0/3343183426 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe01a18e0 con 0x7f5fe0105170 2024-02-05T07:49:42.195 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.191+0000 7f5fe619a700 1 -- 172.21.15.157:0/3343183426 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe01a1de0 con 0x7f5fe0105170 2024-02-05T07:49:42.195 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.192+0000 7f5fe619a700 1 -- 172.21.15.157:0/3343183426 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe00689e0 con 0x7f5fe0105170 2024-02-05T07:49:42.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.192+0000 7f5fdcff9700 1 -- 172.21.15.157:0/3343183426 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f5fd4004750 con 0x7f5fe0105170 2024-02-05T07:49:42.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.193+0000 7f5fdcff9700 1 --2- 172.21.15.157:0/3343183426 >> v2:172.21.15.131:6800/760045032 conn(0x7f5fc806d990 0x7f5fc806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:42.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.193+0000 7f5fdcff9700 1 -- 172.21.15.157:0/3343183426 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4081+0+0 (secure 0 0 0) 0x7f5fd408c9c0 con 0x7f5fe0105170 2024-02-05T07:49:42.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.193+0000 7f5fdf7fe700 1 --2- 172.21.15.157:0/3343183426 >> v2:172.21.15.131:6800/760045032 conn(0x7f5fc806d990 0x7f5fc806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:42.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.193+0000 7f5fdf7fe700 1 --2- 172.21.15.157:0/3343183426 >> v2:172.21.15.131:6800/760045032 conn(0x7f5fc806d990 0x7f5fc806fe50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0005fd0 tx=0x7f5fd0005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:42.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.197+0000 7f5fdcff9700 1 -- 172.21.15.157:0/3343183426 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5fd4056b30 con 0x7f5fe0105170 2024-02-05T07:49:42.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.402+0000 7f5fe619a700 1 -- 172.21.15.157:0/3343183426 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi131=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fe01095a0 con 0x7f5fc806d990 2024-02-05T07:49:42.410 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.410+0000 7f5fdcff9700 1 -- 172.21.15.157:0/3343183426 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7f5fe01095a0 con 0x7f5fc806d990 2024-02-05T07:49:42.411 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled rgw.foo.a update... 2024-02-05T07:49:42.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.412+0000 7f5fe619a700 1 -- 172.21.15.157:0/3343183426 >> v2:172.21.15.131:6800/760045032 conn(0x7f5fc806d990 msgr2=0x7f5fc806fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:42.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.412+0000 7f5fe619a700 1 --2- 172.21.15.157:0/3343183426 >> v2:172.21.15.131:6800/760045032 conn(0x7f5fc806d990 0x7f5fc806fe50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0005fd0 tx=0x7f5fd0005dc0 comp rx=0 tx=0).stop 2024-02-05T07:49:42.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.412+0000 7f5fe619a700 1 -- 172.21.15.157:0/3343183426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe0105170 msgr2=0x7f5fe01007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:42.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.412+0000 7f5fe619a700 1 --2- 172.21.15.157:0/3343183426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe0105170 0x7f5fe01007f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400c920 tx=0x7f5fd400cce0 comp rx=0 tx=0).stop 2024-02-05T07:49:42.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.412+0000 7f5fe619a700 1 -- 172.21.15.157:0/3343183426 shutdown_connections 2024-02-05T07:49:42.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.412+0000 7f5fe619a700 1 --2- 172.21.15.157:0/3343183426 >> v2:172.21.15.131:6800/760045032 conn(0x7f5fc806d990 0x7f5fc806fe50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:42.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.412+0000 7f5fe619a700 1 --2- 172.21.15.157:0/3343183426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fe0102840 0x7f5fe01021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:42.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.412+0000 7f5fe619a700 1 --2- 172.21.15.157:0/3343183426 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f5fe0107aa0 0x7f5fe0100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:42.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.412+0000 7f5fe619a700 1 --2- 172.21.15.157:0/3343183426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe0105170 0x7f5fe01007f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:42.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.412+0000 7f5fe619a700 1 -- 172.21.15.157:0/3343183426 >> 172.21.15.157:0/3343183426 conn(0x7f5fe00fc1f0 msgr2=0x7f5fe0107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:42.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.412+0000 7f5fe619a700 1 -- 172.21.15.157:0/3343183426 shutdown_connections 2024-02-05T07:49:42.413 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:42.412+0000 7f5fe619a700 1 -- 172.21.15.157:0/3343183426 wait complete. 2024-02-05T07:49:43.021 DEBUG:teuthology.orchestra.run.smithi131:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@rgw.foo.a.service 2024-02-05T07:49:43.024 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi157 2024-02-05T07:49:43.024 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph osd pool create datapool 3 3 replicated 2024-02-05T07:49:43.262 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:49:43.639 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:43 smithi157 ceph-mon[33345]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi131=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:49:43.639 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:43 smithi157 ceph-mon[33345]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-02-05T07:49:43.639 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:43 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:43.639 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:43 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:43.640 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:43 smithi157 ceph-mon[33345]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-05T07:49:43.640 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:43 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:43.640 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:43 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:43.640 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:43 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:43.640 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:43 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-05T07:49:43.640 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:43 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-05T07:49:43.640 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:43 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:43.640 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:43 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:43.677 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[29507]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi131=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:49:43.677 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[29507]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-02-05T07:49:43.677 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:43.677 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:43.677 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[29507]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-05T07:49:43.677 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:43.677 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:43.678 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:43.678 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-05T07:49:43.678 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-05T07:49:43.678 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:43.678 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:43.678 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[41654]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi131=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:49:43.678 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[41654]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-02-05T07:49:43.678 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:43.678 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:43.678 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[41654]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-05T07:49:43.678 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:43.678 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:43.678 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:43.678 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-05T07:49:43.679 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-05T07:49:43.679 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:43.679 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:43 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:44.612 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:44 smithi157 ceph-mon[33345]: Deploying daemon rgw.foo.a on smithi131 2024-02-05T07:49:44.650 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:44 smithi131 ceph-mon[41654]: Deploying daemon rgw.foo.a on smithi131 2024-02-05T07:49:44.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:44 smithi131 ceph-mon[29507]: Deploying daemon rgw.foo.a on smithi131 2024-02-05T07:49:44.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.885+0000 7fee6071f700 1 -- 172.21.15.157:0/2484418031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee58102820 msgr2=0x7fee581090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:44.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.885+0000 7fee6071f700 1 --2- 172.21.15.157:0/2484418031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee58102820 0x7fee581090d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fee500099c0 tx=0x7fee50009cd0 comp rx=0 tx=0).stop 2024-02-05T07:49:44.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.885+0000 7fee6071f700 1 -- 172.21.15.157:0/2484418031 shutdown_connections 2024-02-05T07:49:44.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.885+0000 7fee6071f700 1 --2- 172.21.15.157:0/2484418031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee58100c80 0x7fee581010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:44.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.885+0000 7fee6071f700 1 --2- 172.21.15.157:0/2484418031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee58102820 0x7fee581090d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:44.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.885+0000 7fee6071f700 1 --2- 172.21.15.157:0/2484418031 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fee58101e80 0x7fee581022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:44.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.885+0000 7fee6071f700 1 -- 172.21.15.157:0/2484418031 >> 172.21.15.157:0/2484418031 conn(0x7fee580fc240 msgr2=0x7fee580fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:44.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.885+0000 7fee6071f700 1 -- 172.21.15.157:0/2484418031 shutdown_connections 2024-02-05T07:49:44.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.885+0000 7fee6071f700 1 -- 172.21.15.157:0/2484418031 wait complete. 2024-02-05T07:49:44.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.885+0000 7fee6071f700 1 Processor -- start 2024-02-05T07:49:44.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.885+0000 7fee6071f700 1 -- start start 2024-02-05T07:49:44.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee6071f700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fee58100c80 0x7fee58104c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:44.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee6071f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee58101e80 0x7fee58103260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:44.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee6071f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee58102820 0x7fee581037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:44.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee6071f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee58103f20 con 0x7fee58102820 2024-02-05T07:49:44.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee6071f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee58104090 con 0x7fee58101e80 2024-02-05T07:49:44.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee6071f700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee581a13a0 con 0x7fee58100c80 2024-02-05T07:49:44.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee5dcba700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee58101e80 0x7fee58103260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:44.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee5e4bb700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fee58100c80 0x7fee58104c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:44.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee5ecbc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee58102820 0x7fee581037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:44.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee5e4bb700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fee58100c80 0x7fee58104c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.157:34842/0 (socket says 172.21.15.157:34842) 2024-02-05T07:49:44.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee5e4bb700 1 -- 172.21.15.157:0/2726647975 learned_addr learned my addr 172.21.15.157:0/2726647975 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:49:44.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee5e4bb700 1 -- 172.21.15.157:0/2726647975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee58101e80 msgr2=0x7fee58103260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:44.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee5e4bb700 1 --2- 172.21.15.157:0/2726647975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee58101e80 0x7fee58103260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:44.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee5e4bb700 1 -- 172.21.15.157:0/2726647975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee58102820 msgr2=0x7fee581037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:44.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee5e4bb700 1 --2- 172.21.15.157:0/2726647975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee58102820 0x7fee581037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:44.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee5e4bb700 1 -- 172.21.15.157:0/2726647975 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee500095f0 con 0x7fee58100c80 2024-02-05T07:49:44.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee5e4bb700 1 --2- 172.21.15.157:0/2726647975 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fee58100c80 0x7fee58104c10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fee4800efc0 tx=0x7fee4800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:44.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee577fe700 1 -- 172.21.15.157:0/2726647975 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48018070 con 0x7fee58100c80 2024-02-05T07:49:44.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee577fe700 1 -- 172.21.15.157:0/2726647975 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fee4800f040 con 0x7fee58100c80 2024-02-05T07:49:44.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.886+0000 7fee6071f700 1 -- 172.21.15.157:0/2726647975 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee581a1620 con 0x7fee58100c80 2024-02-05T07:49:44.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.887+0000 7fee577fe700 1 -- 172.21.15.157:0/2726647975 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48004280 con 0x7fee58100c80 2024-02-05T07:49:44.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.887+0000 7fee6071f700 1 -- 172.21.15.157:0/2726647975 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee581a1b70 con 0x7fee58100c80 2024-02-05T07:49:44.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.887+0000 7fee6071f700 1 -- 172.21.15.157:0/2726647975 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee58108a60 con 0x7fee58100c80 2024-02-05T07:49:44.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.888+0000 7fee577fe700 1 -- 172.21.15.157:0/2726647975 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7fee48003a00 con 0x7fee58100c80 2024-02-05T07:49:44.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.888+0000 7fee577fe700 1 --2- 172.21.15.157:0/2726647975 >> v2:172.21.15.131:6800/760045032 conn(0x7fee4006d990 0x7fee4006fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:44.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.888+0000 7fee577fe700 1 -- 172.21.15.157:0/2726647975 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4081+0+0 (secure 0 0 0) 0x7fee4808ca00 con 0x7fee58100c80 2024-02-05T07:49:44.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.888+0000 7fee5dcba700 1 --2- 172.21.15.157:0/2726647975 >> v2:172.21.15.131:6800/760045032 conn(0x7fee4006d990 0x7fee4006fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:44.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.889+0000 7fee5dcba700 1 --2- 172.21.15.157:0/2726647975 >> v2:172.21.15.131:6800/760045032 conn(0x7fee4006d990 0x7fee4006fe50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009c60 tx=0x7fee4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:44.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:44.893+0000 7fee577fe700 1 -- 172.21.15.157:0/2726647975 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee48056b50 con 0x7fee58100c80 2024-02-05T07:49:45.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:45.084+0000 7fee6071f700 1 -- 172.21.15.157:0/2726647975 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7fee5810be60 con 0x7fee58100c80 2024-02-05T07:49:45.268 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Feb 05 07:49:45 smithi131 systemd[1]: Starting Ceph rgw.foo.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T07:49:45.421 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:45.420+0000 7fee577fe700 1 -- 172.21.15.157:0/2726647975 <== mon.2 v2:172.21.15.131:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v61) v1 ==== 160+0+0 (secure 0 0 0) 0x7fee4805a210 con 0x7fee58100c80 2024-02-05T07:49:45.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:45.422+0000 7fee6071f700 1 -- 172.21.15.157:0/2726647975 >> v2:172.21.15.131:6800/760045032 conn(0x7fee4006d990 msgr2=0x7fee4006fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:45.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:45.422+0000 7fee6071f700 1 --2- 172.21.15.157:0/2726647975 >> v2:172.21.15.131:6800/760045032 conn(0x7fee4006d990 0x7fee4006fe50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009c60 tx=0x7fee4c009260 comp rx=0 tx=0).stop 2024-02-05T07:49:45.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:45.422+0000 7fee6071f700 1 -- 172.21.15.157:0/2726647975 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fee58100c80 msgr2=0x7fee58104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:45.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:45.422+0000 7fee6071f700 1 --2- 172.21.15.157:0/2726647975 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fee58100c80 0x7fee58104c10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fee4800efc0 tx=0x7fee4800c3a0 comp rx=0 tx=0).stop 2024-02-05T07:49:45.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:45.423+0000 7fee6071f700 1 -- 172.21.15.157:0/2726647975 shutdown_connections 2024-02-05T07:49:45.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:45.423+0000 7fee6071f700 1 --2- 172.21.15.157:0/2726647975 >> v2:172.21.15.131:6800/760045032 conn(0x7fee4006d990 0x7fee4006fe50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:45.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:45.423+0000 7fee6071f700 1 --2- 172.21.15.157:0/2726647975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee58102820 0x7fee581037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:45.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:45.423+0000 7fee6071f700 1 --2- 172.21.15.157:0/2726647975 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fee58100c80 0x7fee58104c10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:45.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:45.423+0000 7fee6071f700 1 --2- 172.21.15.157:0/2726647975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee58101e80 0x7fee58103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:45.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:45.423+0000 7fee6071f700 1 -- 172.21.15.157:0/2726647975 >> 172.21.15.157:0/2726647975 conn(0x7fee580fc240 msgr2=0x7fee580fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:45.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:45.423+0000 7fee6071f700 1 -- 172.21.15.157:0/2726647975 shutdown_connections 2024-02-05T07:49:45.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:45.423+0000 7fee6071f700 1 -- 172.21.15.157:0/2726647975 wait complete. 2024-02-05T07:49:45.425 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'datapool' created 2024-02-05T07:49:45.588 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:45 smithi131 ceph-mon[41654]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-05T07:49:45.588 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.157:0/2726647975' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-05T07:49:45.588 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-05T07:49:45.589 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:45 smithi131 ceph-mon[29507]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-05T07:49:45.589 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.157:0/2726647975' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-05T07:49:45.589 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-05T07:49:45.589 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Feb 05 07:49:45 smithi131 podman[70093]: 2024-02-05 07:49:45.231965435 +0000 UTC m=+0.018292868 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T07:49:45.589 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Feb 05 07:49:45 smithi131 podman[70093]: 2024-02-05 07:49:45.360468005 +0000 UTC m=+0.146795492 container create ecfde129a160d4460804706e652c3a342423b409c7bdd90876df00f2d0c852a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-rgw-foo-a, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131) 2024-02-05T07:49:45.589 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Feb 05 07:49:45 smithi131 podman[70093]: 2024-02-05 07:49:45.588279981 +0000 UTC m=+0.374607409 container init ecfde129a160d4460804706e652c3a342423b409c7bdd90876df00f2d0c852a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-rgw-foo-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2) 2024-02-05T07:49:45.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:45 smithi157 ceph-mon[33345]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-05T07:49:45.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.157:0/2726647975' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-05T07:49:45.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-05T07:49:46.019 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Feb 05 07:49:45 smithi131 podman[70093]: 2024-02-05 07:49:45.593860287 +0000 UTC m=+0.380187718 container start ecfde129a160d4460804706e652c3a342423b409c7bdd90876df00f2d0c852a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-rgw-foo-a, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-02-05T07:49:46.020 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Feb 05 07:49:45 smithi131 bash[70093]: ecfde129a160d4460804706e652c3a342423b409c7bdd90876df00f2d0c852a8 2024-02-05T07:49:46.020 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Feb 05 07:49:45 smithi131 systemd[1]: Started Ceph rgw.foo.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T07:49:46.027 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- rbd pool init datapool 2024-02-05T07:49:46.263 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:49:46.645 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:46 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-05T07:49:46.645 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:46 smithi157 ceph-mon[33345]: osdmap e61: 8 total, 8 up, 8 in 2024-02-05T07:49:46.645 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:46 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:46.645 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:46 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:46.645 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:46 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:46.645 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:46 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:46.645 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:46 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:46.645 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:46 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:46.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:46 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-05T07:49:46.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:46 smithi131 ceph-mon[29507]: osdmap e61: 8 total, 8 up, 8 in 2024-02-05T07:49:46.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:46 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:46.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:46 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:46.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:46 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:46.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:46 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:46.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:46 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:46.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:46 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:46.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:46 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-05T07:49:46.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:46 smithi131 ceph-mon[41654]: osdmap e61: 8 total, 8 up, 8 in 2024-02-05T07:49:46.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:46 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:46.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:46 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:46.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:46 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:46.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:46 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:46.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:46 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:46.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:46 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:47.697 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:47 smithi157 ceph-mon[33345]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-02-05T07:49:47.697 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:47 smithi157 ceph-mon[33345]: osdmap e62: 8 total, 8 up, 8 in 2024-02-05T07:49:47.697 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:47 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1428962963' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-05T07:49:47.697 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-05T07:49:47.697 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:47 smithi157 ceph-mon[33345]: pgmap v190: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:49:47.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:47 smithi131 ceph-mon[41654]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-02-05T07:49:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:47 smithi131 ceph-mon[41654]: osdmap e62: 8 total, 8 up, 8 in 2024-02-05T07:49:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:47 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1428962963' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-05T07:49:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-05T07:49:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:47 smithi131 ceph-mon[41654]: pgmap v190: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:49:47.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:47 smithi131 ceph-mon[29507]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-02-05T07:49:47.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:47 smithi131 ceph-mon[29507]: osdmap e62: 8 total, 8 up, 8 in 2024-02-05T07:49:47.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:47 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1428962963' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-05T07:49:47.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-05T07:49:47.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:47 smithi131 ceph-mon[29507]: pgmap v190: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:49:48.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:48 smithi157 ceph-mon[33345]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:48.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-05T07:49:48.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:48 smithi157 ceph-mon[33345]: osdmap e63: 8 total, 8 up, 8 in 2024-02-05T07:49:48.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:48 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:48.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:48 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:48.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:48 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:48.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:48 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:48.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.157:0/2990167474' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-05T07:49:48.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:48 smithi131 ceph-mon[29507]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:48.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-05T07:49:48.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:48 smithi131 ceph-mon[29507]: osdmap e63: 8 total, 8 up, 8 in 2024-02-05T07:49:48.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:48 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:48.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:48 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:48.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:48 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:48.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:48 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:48.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.157:0/2990167474' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-05T07:49:48.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:48 smithi131 ceph-mon[41654]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:48.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-05T07:49:48.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:48 smithi131 ceph-mon[41654]: osdmap e63: 8 total, 8 up, 8 in 2024-02-05T07:49:48.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:48 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:48.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:48 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:48.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:48 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:48.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:48 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:48.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.157:0/2990167474' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-05T07:49:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:49 smithi157 ceph-mon[33345]: Checking dashboard <-> RGW credentials 2024-02-05T07:49:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.157:0/2990167474' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-05T07:49:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:49 smithi157 ceph-mon[33345]: osdmap e64: 8 total, 8 up, 8 in 2024-02-05T07:49:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T07:49:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T07:49:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:49 smithi157 ceph-mon[33345]: pgmap v193: 68 pgs: 14 creating+peering, 5 active+clean, 49 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:49:50.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T07:49:50.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T07:49:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:49 smithi131 ceph-mon[29507]: Checking dashboard <-> RGW credentials 2024-02-05T07:49:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.157:0/2990167474' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-05T07:49:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:49 smithi131 ceph-mon[29507]: osdmap e64: 8 total, 8 up, 8 in 2024-02-05T07:49:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T07:49:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T07:49:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:49 smithi131 ceph-mon[29507]: pgmap v193: 68 pgs: 14 creating+peering, 5 active+clean, 49 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:49:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T07:49:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T07:49:50.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:49 smithi131 ceph-mon[41654]: Checking dashboard <-> RGW credentials 2024-02-05T07:49:50.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.157:0/2990167474' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-05T07:49:50.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:49 smithi131 ceph-mon[41654]: osdmap e64: 8 total, 8 up, 8 in 2024-02-05T07:49:50.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T07:49:50.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T07:49:50.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:49 smithi131 ceph-mon[41654]: pgmap v193: 68 pgs: 14 creating+peering, 5 active+clean, 49 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:49:50.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T07:49:50.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T07:49:51.071 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:50 smithi157 ceph-mon[33345]: osdmap e65: 8 total, 8 up, 8 in 2024-02-05T07:49:51.072 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:50 smithi157 ceph-mon[33345]: osdmap e66: 8 total, 8 up, 8 in 2024-02-05T07:49:51.072 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T07:49:51.072 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T07:49:51.072 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:50 smithi157 ceph-mon[33345]: pgmap v196: 100 pgs: 21 creating+peering, 18 active+clean, 61 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:49:51.072 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:50 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:51.103 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.157 --placement '1;smithi157=iscsi.a' 2024-02-05T07:49:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:50 smithi131 ceph-mon[29507]: osdmap e65: 8 total, 8 up, 8 in 2024-02-05T07:49:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:50 smithi131 ceph-mon[29507]: osdmap e66: 8 total, 8 up, 8 in 2024-02-05T07:49:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T07:49:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T07:49:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:50 smithi131 ceph-mon[29507]: pgmap v196: 100 pgs: 21 creating+peering, 18 active+clean, 61 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:49:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:50 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:51.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:50 smithi131 ceph-mon[41654]: osdmap e65: 8 total, 8 up, 8 in 2024-02-05T07:49:51.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:50 smithi131 ceph-mon[41654]: osdmap e66: 8 total, 8 up, 8 in 2024-02-05T07:49:51.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T07:49:51.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T07:49:51.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:50 smithi131 ceph-mon[41654]: pgmap v196: 100 pgs: 21 creating+peering, 18 active+clean, 61 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:49:51.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:50 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:51.346 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:49:52.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T07:49:52.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T07:49:52.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:52 smithi157 ceph-mon[33345]: osdmap e67: 8 total, 8 up, 8 in 2024-02-05T07:49:52.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T07:49:52.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T07:49:52.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:52 smithi131 ceph-mon[41654]: osdmap e67: 8 total, 8 up, 8 in 2024-02-05T07:49:52.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T07:49:52.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T07:49:52.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:52 smithi131 ceph-mon[29507]: osdmap e67: 8 total, 8 up, 8 in 2024-02-05T07:49:52.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.961+0000 7f1d31f61700 1 -- 172.21.15.157:0/223663119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d2c1003e0 msgr2=0x7f1d2c100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:52.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.961+0000 7f1d31f61700 1 --2- 172.21.15.157:0/223663119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d2c1003e0 0x7f1d2c100800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d200099c0 tx=0x7f1d20009cd0 comp rx=0 tx=0).stop 2024-02-05T07:49:52.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.961+0000 7f1d31f61700 1 -- 172.21.15.157:0/223663119 shutdown_connections 2024-02-05T07:49:52.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.961+0000 7f1d31f61700 1 --2- 172.21.15.157:0/223663119 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1d2c101f80 0x7f1d2c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:52.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.961+0000 7f1d31f61700 1 --2- 172.21.15.157:0/223663119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d2c1003e0 0x7f1d2c100800 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:52.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.961+0000 7f1d31f61700 1 --2- 172.21.15.157:0/223663119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d2c1015e0 0x7f1d2c101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:52.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.961+0000 7f1d31f61700 1 -- 172.21.15.157:0/223663119 >> 172.21.15.157:0/223663119 conn(0x7f1d2c0fb9c0 msgr2=0x7f1d2c0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:52.968 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.961+0000 7f1d31f61700 1 -- 172.21.15.157:0/223663119 shutdown_connections 2024-02-05T07:49:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.961+0000 7f1d31f61700 1 -- 172.21.15.157:0/223663119 wait complete. 2024-02-05T07:49:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.961+0000 7f1d31f61700 1 Processor -- start 2024-02-05T07:49:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.961+0000 7f1d31f61700 1 -- start start 2024-02-05T07:49:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d31f61700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1d2c1003e0 0x7f1d2c197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d31f61700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d2c1015e0 0x7f1d2c197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d31f61700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d2c101f80 0x7f1d2c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d31f61700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c19ef30 con 0x7f1d2c101f80 2024-02-05T07:49:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d31f61700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c19f070 con 0x7f1d2c1015e0 2024-02-05T07:49:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d31f61700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c19f200 con 0x7f1d2c1003e0 2024-02-05T07:49:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d2affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d2c1015e0 0x7f1d2c197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d2affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d2c1015e0 0x7f1d2c197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43542/0 (socket says 172.21.15.157:43542) 2024-02-05T07:49:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d2affd700 1 -- 172.21.15.157:0/3401464675 learned_addr learned my addr 172.21.15.157:0/3401464675 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:49:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d2b7fe700 1 --2- 172.21.15.157:0/3401464675 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1d2c1003e0 0x7f1d2c197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d2bfff700 1 --2- 172.21.15.157:0/3401464675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d2c101f80 0x7f1d2c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d2affd700 1 -- 172.21.15.157:0/3401464675 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1d2c1003e0 msgr2=0x7f1d2c197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d2affd700 1 --2- 172.21.15.157:0/3401464675 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1d2c1003e0 0x7f1d2c197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d2affd700 1 -- 172.21.15.157:0/3401464675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d2c101f80 msgr2=0x7f1d2c19e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d2affd700 1 --2- 172.21.15.157:0/3401464675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d2c101f80 0x7f1d2c19e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d2affd700 1 -- 172.21.15.157:0/3401464675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d200095f0 con 0x7f1d2c1015e0 2024-02-05T07:49:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d2affd700 1 --2- 172.21.15.157:0/3401464675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d2c1015e0 0x7f1d2c197950 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00c920 tx=0x7f1d1c00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d28ff9700 1 -- 172.21.15.157:0/3401464675 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c004a30 con 0x7f1d2c1015e0 2024-02-05T07:49:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d28ff9700 1 -- 172.21.15.157:0/3401464675 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f1d1c0054d0 con 0x7f1d2c1015e0 2024-02-05T07:49:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d31f61700 1 -- 172.21.15.157:0/3401464675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2c19f4e0 con 0x7f1d2c1015e0 2024-02-05T07:49:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d28ff9700 1 -- 172.21.15.157:0/3401464675 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c007bc0 con 0x7f1d2c1015e0 2024-02-05T07:49:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.962+0000 7f1d31f61700 1 -- 172.21.15.157:0/3401464675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2c19f9e0 con 0x7f1d2c1015e0 2024-02-05T07:49:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.963+0000 7f1d31f61700 1 -- 172.21.15.157:0/3401464675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d2c1081c0 con 0x7f1d2c1015e0 2024-02-05T07:49:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.964+0000 7f1d28ff9700 1 -- 172.21.15.157:0/3401464675 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f1d1c007d20 con 0x7f1d2c1015e0 2024-02-05T07:49:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.964+0000 7f1d28ff9700 1 --2- 172.21.15.157:0/3401464675 >> v2:172.21.15.131:6800/760045032 conn(0x7f1d1406da30 0x7f1d1406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.965+0000 7f1d28ff9700 1 -- 172.21.15.157:0/3401464675 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5915+0+0 (secure 0 0 0) 0x7f1d1c08cc50 con 0x7f1d2c1015e0 2024-02-05T07:49:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.965+0000 7f1d2b7fe700 1 --2- 172.21.15.157:0/3401464675 >> v2:172.21.15.131:6800/760045032 conn(0x7f1d1406da30 0x7f1d1406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:52.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.965+0000 7f1d2b7fe700 1 --2- 172.21.15.157:0/3401464675 >> v2:172.21.15.131:6800/760045032 conn(0x7f1d1406da30 0x7f1d1406fef0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1d20006010 tx=0x7f1d20003760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:52.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:52.969+0000 7f1d28ff9700 1 -- 172.21.15.157:0/3401464675 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1d1c092020 con 0x7f1d2c1015e0 2024-02-05T07:49:53.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:53.182+0000 7f1d31f61700 1 -- 172.21.15.157:0/3401464675 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.157", "placement": "1;smithi157=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d2c062160 con 0x7f1d1406da30 2024-02-05T07:49:53.191 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:53.190+0000 7f1d28ff9700 1 -- 172.21.15.157:0/3401464675 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (secure 0 0 0) 0x7f1d2c062160 con 0x7f1d1406da30 2024-02-05T07:49:53.191 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled iscsi.datapool update... 2024-02-05T07:49:53.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:53.192+0000 7f1d31f61700 1 -- 172.21.15.157:0/3401464675 >> v2:172.21.15.131:6800/760045032 conn(0x7f1d1406da30 msgr2=0x7f1d1406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:53.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:53.192+0000 7f1d31f61700 1 --2- 172.21.15.157:0/3401464675 >> v2:172.21.15.131:6800/760045032 conn(0x7f1d1406da30 0x7f1d1406fef0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1d20006010 tx=0x7f1d20003760 comp rx=0 tx=0).stop 2024-02-05T07:49:53.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:53.192+0000 7f1d31f61700 1 -- 172.21.15.157:0/3401464675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d2c1015e0 msgr2=0x7f1d2c197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:53.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:53.192+0000 7f1d31f61700 1 --2- 172.21.15.157:0/3401464675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d2c1015e0 0x7f1d2c197950 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00c920 tx=0x7f1d1c00cc30 comp rx=0 tx=0).stop 2024-02-05T07:49:53.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:53.192+0000 7f1d31f61700 1 -- 172.21.15.157:0/3401464675 shutdown_connections 2024-02-05T07:49:53.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:53.192+0000 7f1d31f61700 1 --2- 172.21.15.157:0/3401464675 >> v2:172.21.15.131:6800/760045032 conn(0x7f1d1406da30 0x7f1d1406fef0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:53.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:53.192+0000 7f1d31f61700 1 --2- 172.21.15.157:0/3401464675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d2c101f80 0x7f1d2c19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:53.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:53.192+0000 7f1d31f61700 1 --2- 172.21.15.157:0/3401464675 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1d2c1003e0 0x7f1d2c197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:53.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:53.192+0000 7f1d31f61700 1 --2- 172.21.15.157:0/3401464675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d2c1015e0 0x7f1d2c197950 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:53.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:53.192+0000 7f1d31f61700 1 -- 172.21.15.157:0/3401464675 >> 172.21.15.157:0/3401464675 conn(0x7f1d2c0fb9c0 msgr2=0x7f1d2c0fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:53.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:53.193+0000 7f1d31f61700 1 -- 172.21.15.157:0/3401464675 shutdown_connections 2024-02-05T07:49:53.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:53.193+0000 7f1d31f61700 1 -- 172.21.15.157:0/3401464675 wait complete. 2024-02-05T07:49:53.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:53 smithi157 ceph-mon[33345]: osdmap e68: 8 total, 8 up, 8 in 2024-02-05T07:49:53.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T07:49:53.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T07:49:53.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:53 smithi157 ceph-mon[33345]: pgmap v199: 132 pgs: 11 creating+peering, 75 active+clean, 46 unknown; 578 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-02-05T07:49:53.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:53 smithi157 ceph-mon[33345]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:53.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:53 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:53.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:53 smithi131 ceph-mon[41654]: osdmap e68: 8 total, 8 up, 8 in 2024-02-05T07:49:53.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T07:49:53.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T07:49:53.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:53 smithi131 ceph-mon[41654]: pgmap v199: 132 pgs: 11 creating+peering, 75 active+clean, 46 unknown; 578 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-02-05T07:49:53.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:53 smithi131 ceph-mon[41654]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:53.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:53 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:53.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:53 smithi131 ceph-mon[29507]: osdmap e68: 8 total, 8 up, 8 in 2024-02-05T07:49:53.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T07:49:53.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T07:49:53.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:53 smithi131 ceph-mon[29507]: pgmap v199: 132 pgs: 11 creating+peering, 75 active+clean, 46 unknown; 578 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-02-05T07:49:53.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:53 smithi131 ceph-mon[29507]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:53.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:53 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:53.779 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-02-05T07:49:53.779 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:49:53.779 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-02-05T07:49:53.815 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-02-05T07:49:53.815 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-02-05T07:49:53.848 DEBUG:teuthology.orchestra.run.smithi157:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@iscsi.iscsi.a.service 2024-02-05T07:49:53.891 INFO:tasks.cephadm:Adding prometheus.a on smithi157 2024-02-05T07:49:53.891 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch apply prometheus '1;smithi157=a' 2024-02-05T07:49:54.167 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:49:54.589 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:54 smithi157 ceph-mon[33345]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.157", "placement": "1;smithi157=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:49:54.589 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:54 smithi157 ceph-mon[33345]: Saving service iscsi.datapool spec with placement smithi157=iscsi.a;count:1 2024-02-05T07:49:54.589 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T07:49:54.589 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T07:49:54.589 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:54 smithi157 ceph-mon[33345]: osdmap e69: 8 total, 8 up, 8 in 2024-02-05T07:49:54.589 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T07:49:54.589 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T07:49:54.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:54 smithi131 ceph-mon[29507]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.157", "placement": "1;smithi157=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:49:54.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:54 smithi131 ceph-mon[29507]: Saving service iscsi.datapool spec with placement smithi157=iscsi.a;count:1 2024-02-05T07:49:54.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T07:49:54.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T07:49:54.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:54 smithi131 ceph-mon[29507]: osdmap e69: 8 total, 8 up, 8 in 2024-02-05T07:49:54.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T07:49:54.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T07:49:54.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:54 smithi131 ceph-mon[41654]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.157", "placement": "1;smithi157=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:49:54.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:54 smithi131 ceph-mon[41654]: Saving service iscsi.datapool spec with placement smithi157=iscsi.a;count:1 2024-02-05T07:49:54.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T07:49:54.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T07:49:54.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:54 smithi131 ceph-mon[41654]: osdmap e69: 8 total, 8 up, 8 in 2024-02-05T07:49:54.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T07:49:54.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T07:49:55.643 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T07:49:55.643 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T07:49:55.643 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:55 smithi157 ceph-mon[33345]: osdmap e70: 8 total, 8 up, 8 in 2024-02-05T07:49:55.643 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:55 smithi157 ceph-mon[33345]: pgmap v202: 132 pgs: 4 creating+activating, 6 creating+peering, 95 active+clean, 27 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-05T07:49:55.643 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:55.643 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:55.643 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:55.643 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-05T07:49:55.643 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-05T07:49:55.643 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:55 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:55.644 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:55 smithi157 ceph-mon[33345]: Deploying daemon iscsi.iscsi.a on smithi157 2024-02-05T07:49:55.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T07:49:55.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T07:49:55.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[41654]: osdmap e70: 8 total, 8 up, 8 in 2024-02-05T07:49:55.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[41654]: pgmap v202: 132 pgs: 4 creating+activating, 6 creating+peering, 95 active+clean, 27 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-05T07:49:55.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:55.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:55.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:55.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-05T07:49:55.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-05T07:49:55.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:55.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[41654]: Deploying daemon iscsi.iscsi.a on smithi157 2024-02-05T07:49:55.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3405383090' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T07:49:55.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2126927015' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T07:49:55.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[29507]: osdmap e70: 8 total, 8 up, 8 in 2024-02-05T07:49:55.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[29507]: pgmap v202: 132 pgs: 4 creating+activating, 6 creating+peering, 95 active+clean, 27 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-05T07:49:55.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:55.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:55.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:55.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-05T07:49:55.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-05T07:49:55.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:55.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:55 smithi131 ceph-mon[29507]: Deploying daemon iscsi.iscsi.a on smithi157 2024-02-05T07:49:55.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.858+0000 7fb104dcf700 1 -- 172.21.15.157:0/2788070419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100102840 msgr2=0x7fb1001090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:55.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.858+0000 7fb104dcf700 1 --2- 172.21.15.157:0/2788070419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100102840 0x7fb1001090f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800b0d0 tx=0x7fb0f800b3e0 comp rx=0 tx=0).stop 2024-02-05T07:49:55.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.858+0000 7fb104dcf700 1 -- 172.21.15.157:0/2788070419 shutdown_connections 2024-02-05T07:49:55.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.858+0000 7fb104dcf700 1 --2- 172.21.15.157:0/2788070419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100102840 0x7fb1001090f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:55.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.858+0000 7fb104dcf700 1 --2- 172.21.15.157:0/2788070419 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fb100100ca0 0x7fb1001010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:55.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.858+0000 7fb104dcf700 1 --2- 172.21.15.157:0/2788070419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb100101ea0 0x7fb100102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:55.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.858+0000 7fb104dcf700 1 -- 172.21.15.157:0/2788070419 >> 172.21.15.157:0/2788070419 conn(0x7fb1000fc220 msgr2=0x7fb1000fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:55.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.858+0000 7fb104dcf700 1 -- 172.21.15.157:0/2788070419 shutdown_connections 2024-02-05T07:49:55.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.858+0000 7fb104dcf700 1 -- 172.21.15.157:0/2788070419 wait complete. 2024-02-05T07:49:55.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb104dcf700 1 Processor -- start 2024-02-05T07:49:55.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb104dcf700 1 -- start start 2024-02-05T07:49:55.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb104dcf700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fb100100ca0 0x7fb10006af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:55.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb104dcf700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb100101ea0 0x7fb10019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:55.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb104dcf700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100102840 0x7fb1001a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:55.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb104dcf700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1001a1190 con 0x7fb100101ea0 2024-02-05T07:49:55.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb104dcf700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1001a1300 con 0x7fb100102840 2024-02-05T07:49:55.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb104dcf700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1001a14c0 con 0x7fb100100ca0 2024-02-05T07:49:55.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb0fed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100102840 0x7fb1001a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:55.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb0fed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100102840 0x7fb1001a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43556/0 (socket says 172.21.15.157:43556) 2024-02-05T07:49:55.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb0fed9d700 1 -- 172.21.15.157:0/1588548737 learned_addr learned my addr 172.21.15.157:0/1588548737 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:49:55.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb0fe59c700 1 --2- 172.21.15.157:0/1588548737 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fb100100ca0 0x7fb10006af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:55.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb0fdd9b700 1 --2- 172.21.15.157:0/1588548737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb100101ea0 0x7fb10019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:55.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb0fed9d700 1 -- 172.21.15.157:0/1588548737 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fb100100ca0 msgr2=0x7fb10006af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:55.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb0fed9d700 1 --2- 172.21.15.157:0/1588548737 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fb100100ca0 0x7fb10006af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:55.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb0fed9d700 1 -- 172.21.15.157:0/1588548737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb100101ea0 msgr2=0x7fb10019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:55.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb0fed9d700 1 --2- 172.21.15.157:0/1588548737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb100101ea0 0x7fb10019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:55.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb0fed9d700 1 -- 172.21.15.157:0/1588548737 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f8009d00 con 0x7fb100102840 2024-02-05T07:49:55.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb0fed9d700 1 --2- 172.21.15.157:0/1588548737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100102840 0x7fb1001a0b70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009850 tx=0x7fb0f800cfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:55.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.859+0000 7fb0ef7fe700 1 -- 172.21.15.157:0/1588548737 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800e040 con 0x7fb100102840 2024-02-05T07:49:55.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.860+0000 7fb0ef7fe700 1 -- 172.21.15.157:0/1588548737 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fb0f8009460 con 0x7fb100102840 2024-02-05T07:49:55.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.860+0000 7fb104dcf700 1 -- 172.21.15.157:0/1588548737 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1001a1740 con 0x7fb100102840 2024-02-05T07:49:55.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.860+0000 7fb0ef7fe700 1 -- 172.21.15.157:0/1588548737 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f801b6d0 con 0x7fb100102840 2024-02-05T07:49:55.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.860+0000 7fb104dcf700 1 -- 172.21.15.157:0/1588548737 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1001a1b00 con 0x7fb100102840 2024-02-05T07:49:55.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.861+0000 7fb104dcf700 1 -- 172.21.15.157:0/1588548737 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb100107880 con 0x7fb100102840 2024-02-05T07:49:55.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.861+0000 7fb0ef7fe700 1 -- 172.21.15.157:0/1588548737 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7fb0f8005070 con 0x7fb100102840 2024-02-05T07:49:55.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.861+0000 7fb0ef7fe700 1 --2- 172.21.15.157:0/1588548737 >> v2:172.21.15.131:6800/760045032 conn(0x7fb0e806d990 0x7fb0e806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:55.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.861+0000 7fb0ef7fe700 1 -- 172.21.15.157:0/1588548737 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5942+0+0 (secure 0 0 0) 0x7fb0f808e560 con 0x7fb100102840 2024-02-05T07:49:55.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.862+0000 7fb0fe59c700 1 --2- 172.21.15.157:0/1588548737 >> v2:172.21.15.131:6800/760045032 conn(0x7fb0e806d990 0x7fb0e806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:55.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.862+0000 7fb0fe59c700 1 --2- 172.21.15.157:0/1588548737 >> v2:172.21.15.131:6800/760045032 conn(0x7fb0e806d990 0x7fb0e806fe50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb100101d00 tx=0x7fb0f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:55.866 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:55.866+0000 7fb0ef7fe700 1 -- 172.21.15.157:0/1588548737 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb0f8057f90 con 0x7fb100102840 2024-02-05T07:49:56.076 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:56.076+0000 7fb104dcf700 1 -- 172.21.15.157:0/1588548737 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi157=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fb100062160 con 0x7fb0e806d990 2024-02-05T07:49:56.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:56.083+0000 7fb0ef7fe700 1 -- 172.21.15.157:0/1588548737 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fb100062160 con 0x7fb0e806d990 2024-02-05T07:49:56.084 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled prometheus update... 2024-02-05T07:49:56.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:56.086+0000 7fb104dcf700 1 -- 172.21.15.157:0/1588548737 >> v2:172.21.15.131:6800/760045032 conn(0x7fb0e806d990 msgr2=0x7fb0e806fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:56.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:56.086+0000 7fb104dcf700 1 --2- 172.21.15.157:0/1588548737 >> v2:172.21.15.131:6800/760045032 conn(0x7fb0e806d990 0x7fb0e806fe50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb100101d00 tx=0x7fb0f000b410 comp rx=0 tx=0).stop 2024-02-05T07:49:56.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:56.086+0000 7fb104dcf700 1 -- 172.21.15.157:0/1588548737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100102840 msgr2=0x7fb1001a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:56.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:56.086+0000 7fb104dcf700 1 --2- 172.21.15.157:0/1588548737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100102840 0x7fb1001a0b70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009850 tx=0x7fb0f800cfa0 comp rx=0 tx=0).stop 2024-02-05T07:49:56.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:56.086+0000 7fb104dcf700 1 -- 172.21.15.157:0/1588548737 shutdown_connections 2024-02-05T07:49:56.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:56.086+0000 7fb104dcf700 1 --2- 172.21.15.157:0/1588548737 >> v2:172.21.15.131:6800/760045032 conn(0x7fb0e806d990 0x7fb0e806fe50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:56.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:56.086+0000 7fb104dcf700 1 --2- 172.21.15.157:0/1588548737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb100102840 0x7fb1001a0b70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:56.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:56.086+0000 7fb104dcf700 1 --2- 172.21.15.157:0/1588548737 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fb100100ca0 0x7fb10006af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:56.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:56.086+0000 7fb104dcf700 1 --2- 172.21.15.157:0/1588548737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb100101ea0 0x7fb10019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:56.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:56.086+0000 7fb104dcf700 1 -- 172.21.15.157:0/1588548737 >> 172.21.15.157:0/1588548737 conn(0x7fb1000fc220 msgr2=0x7fb1000fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:56.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:56.086+0000 7fb104dcf700 1 -- 172.21.15.157:0/1588548737 shutdown_connections 2024-02-05T07:49:56.087 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:56.086+0000 7fb104dcf700 1 -- 172.21.15.157:0/1588548737 wait complete. 2024-02-05T07:49:56.644 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:56 smithi157 ceph-mon[33345]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-05T07:49:56.644 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:56 smithi157 ceph-mon[33345]: Cluster is now healthy 2024-02-05T07:49:56.644 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:56 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:56.645 DEBUG:teuthology.orchestra.run.smithi157:prometheus.a> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@prometheus.a.service 2024-02-05T07:49:56.647 INFO:tasks.cephadm:Adding node-exporter.a on smithi131 2024-02-05T07:49:56.647 INFO:tasks.cephadm:Adding node-exporter.b on smithi157 2024-02-05T07:49:56.647 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch apply node-exporter '2;smithi131=a;smithi157=b' 2024-02-05T07:49:56.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:56 smithi131 ceph-mon[41654]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-05T07:49:56.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:56 smithi131 ceph-mon[41654]: Cluster is now healthy 2024-02-05T07:49:56.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:56 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:56.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:56 smithi131 ceph-mon[29507]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-05T07:49:56.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:56 smithi131 ceph-mon[29507]: Cluster is now healthy 2024-02-05T07:49:56.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:56 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:49:57.586 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:49:57.707 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:57 smithi157 ceph-mon[33345]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi157=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:49:57.708 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:57 smithi157 ceph-mon[33345]: Saving service prometheus spec with placement smithi157=a;count:1 2024-02-05T07:49:57.708 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:57 smithi157 ceph-mon[33345]: pgmap v203: 132 pgs: 4 creating+activating, 2 creating+peering, 114 active+clean, 12 unknown; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 25 KiB/s rd, 3.3 KiB/s wr, 60 op/s 2024-02-05T07:49:57.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:57 smithi131 ceph-mon[41654]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi157=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:49:57.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:57 smithi131 ceph-mon[41654]: Saving service prometheus spec with placement smithi157=a;count:1 2024-02-05T07:49:57.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:57 smithi131 ceph-mon[41654]: pgmap v203: 132 pgs: 4 creating+activating, 2 creating+peering, 114 active+clean, 12 unknown; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 25 KiB/s rd, 3.3 KiB/s wr, 60 op/s 2024-02-05T07:49:57.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:57 smithi131 ceph-mon[29507]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi157=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:49:57.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:57 smithi131 ceph-mon[29507]: Saving service prometheus spec with placement smithi157=a;count:1 2024-02-05T07:49:57.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:57 smithi131 ceph-mon[29507]: pgmap v203: 132 pgs: 4 creating+activating, 2 creating+peering, 114 active+clean, 12 unknown; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 25 KiB/s rd, 3.3 KiB/s wr, 60 op/s 2024-02-05T07:49:59.062 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:58 smithi157 ceph-mon[33345]: osdmap e71: 8 total, 8 up, 8 in 2024-02-05T07:49:59.062 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:49:58 smithi157 ceph-mon[33345]: pgmap v205: 132 pgs: 4 creating+activating, 128 active+clean; 581 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s rd, 4.7 KiB/s wr, 149 op/s 2024-02-05T07:49:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:58 smithi131 ceph-mon[29507]: osdmap e71: 8 total, 8 up, 8 in 2024-02-05T07:49:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:49:58 smithi131 ceph-mon[29507]: pgmap v205: 132 pgs: 4 creating+activating, 128 active+clean; 581 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s rd, 4.7 KiB/s wr, 149 op/s 2024-02-05T07:49:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:58 smithi131 ceph-mon[41654]: osdmap e71: 8 total, 8 up, 8 in 2024-02-05T07:49:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:49:58 smithi131 ceph-mon[41654]: pgmap v205: 132 pgs: 4 creating+activating, 128 active+clean; 581 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s rd, 4.7 KiB/s wr, 149 op/s 2024-02-05T07:49:59.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.290+0000 7f3301c31700 1 -- 172.21.15.157:0/2814916334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc1027e0 msgr2=0x7f32fc104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:59.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.290+0000 7f3301c31700 1 --2- 172.21.15.157:0/2814916334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc1027e0 0x7f32fc104bd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009990 tx=0x7f32f4009ca0 comp rx=0 tx=0).stop 2024-02-05T07:49:59.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.290+0000 7f3301c31700 1 -- 172.21.15.157:0/2814916334 shutdown_connections 2024-02-05T07:49:59.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.290+0000 7f3301c31700 1 --2- 172.21.15.157:0/2814916334 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f32fc107a40 0x7f32fc109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:59.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.290+0000 7f3301c31700 1 --2- 172.21.15.157:0/2814916334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc1027e0 0x7f32fc104bd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:59.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.290+0000 7f3301c31700 1 --2- 172.21.15.157:0/2814916334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32fc105110 0x7f32fc107500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:59.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.290+0000 7f3301c31700 1 -- 172.21.15.157:0/2814916334 >> 172.21.15.157:0/2814916334 conn(0x7f32fc0fc1f0 msgr2=0x7f32fc0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:59.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.290+0000 7f3301c31700 1 -- 172.21.15.157:0/2814916334 shutdown_connections 2024-02-05T07:49:59.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.290+0000 7f3301c31700 1 -- 172.21.15.157:0/2814916334 wait complete. 2024-02-05T07:49:59.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.290+0000 7f3301c31700 1 Processor -- start 2024-02-05T07:49:59.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f3301c31700 1 -- start start 2024-02-05T07:49:59.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f3301c31700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f32fc1027e0 0x7f32fc197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:59.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f3301c31700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc105110 0x7f32fc1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:59.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f3301c31700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32fc107a40 0x7f32fc19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:59.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f3301c31700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc19eec0 con 0x7f32fc107a40 2024-02-05T07:49:59.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f3301c31700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc19f000 con 0x7f32fc105110 2024-02-05T07:49:59.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f3301c31700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc19f190 con 0x7f32fc1027e0 2024-02-05T07:49:59.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f32faffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc105110 0x7f32fc1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:59.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f32faffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc105110 0x7f32fc1978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45148/0 (socket says 172.21.15.157:45148) 2024-02-05T07:49:59.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f32faffd700 1 -- 172.21.15.157:0/2657115457 learned_addr learned my addr 172.21.15.157:0/2657115457 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:49:59.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f32fb7fe700 1 --2- 172.21.15.157:0/2657115457 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f32fc1027e0 0x7f32fc197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:59.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f32fbfff700 1 --2- 172.21.15.157:0/2657115457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32fc107a40 0x7f32fc19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:59.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f32faffd700 1 -- 172.21.15.157:0/2657115457 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f32fc1027e0 msgr2=0x7f32fc197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:59.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f32faffd700 1 --2- 172.21.15.157:0/2657115457 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f32fc1027e0 0x7f32fc197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:59.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f32faffd700 1 -- 172.21.15.157:0/2657115457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32fc107a40 msgr2=0x7f32fc19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:59.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f32faffd700 1 --2- 172.21.15.157:0/2657115457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32fc107a40 0x7f32fc19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:59.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f32faffd700 1 -- 172.21.15.157:0/2657115457 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f40095f0 con 0x7f32fc105110 2024-02-05T07:49:59.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.291+0000 7f32faffd700 1 --2- 172.21.15.157:0/2657115457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc105110 0x7f32fc1978c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32f000c9b0 tx=0x7f32f000ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:59.305 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.292+0000 7f32f8ff9700 1 -- 172.21.15.157:0/2657115457 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0007b00 con 0x7f32fc105110 2024-02-05T07:49:59.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.292+0000 7f32f8ff9700 1 -- 172.21.15.157:0/2657115457 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f32f000ce50 con 0x7f32fc105110 2024-02-05T07:49:59.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.292+0000 7f3301c31700 1 -- 172.21.15.157:0/2657115457 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32fc19f470 con 0x7f32fc105110 2024-02-05T07:49:59.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.292+0000 7f32f8ff9700 1 -- 172.21.15.157:0/2657115457 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0018660 con 0x7f32fc105110 2024-02-05T07:49:59.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.292+0000 7f3301c31700 1 -- 172.21.15.157:0/2657115457 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32fc19f970 con 0x7f32fc105110 2024-02-05T07:49:59.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.293+0000 7f3301c31700 1 -- 172.21.15.157:0/2657115457 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32fc0689e0 con 0x7f32fc105110 2024-02-05T07:49:59.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.293+0000 7f32f8ff9700 1 -- 172.21.15.157:0/2657115457 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f32f000f450 con 0x7f32fc105110 2024-02-05T07:49:59.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.294+0000 7f32f8ff9700 1 --2- 172.21.15.157:0/2657115457 >> v2:172.21.15.131:6800/760045032 conn(0x7f32e406d990 0x7f32e406fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:49:59.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.294+0000 7f32f8ff9700 1 -- 172.21.15.157:0/2657115457 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5942+0+0 (secure 0 0 0) 0x7f32f008d0c0 con 0x7f32fc105110 2024-02-05T07:49:59.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.294+0000 7f32fb7fe700 1 --2- 172.21.15.157:0/2657115457 >> v2:172.21.15.131:6800/760045032 conn(0x7f32e406d990 0x7f32e406fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:49:59.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.294+0000 7f32fb7fe700 1 --2- 172.21.15.157:0/2657115457 >> v2:172.21.15.131:6800/760045032 conn(0x7f32e406d990 0x7f32e406fe50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f32f400b5c0 tx=0x7f32f40037d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:49:59.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.298+0000 7f32f8ff9700 1 -- 172.21.15.157:0/2657115457 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f32f0056af0 con 0x7f32fc105110 2024-02-05T07:49:59.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.511+0000 7f3301c31700 1 -- 172.21.15.157:0/2657115457 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi131=a;smithi157=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f32fc1095c0 con 0x7f32e406d990 2024-02-05T07:49:59.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.518+0000 7f32f8ff9700 1 -- 172.21.15.157:0/2657115457 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f32fc1095c0 con 0x7f32e406d990 2024-02-05T07:49:59.519 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled node-exporter update... 2024-02-05T07:49:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.520+0000 7f3301c31700 1 -- 172.21.15.157:0/2657115457 >> v2:172.21.15.131:6800/760045032 conn(0x7f32e406d990 msgr2=0x7f32e406fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.520+0000 7f3301c31700 1 --2- 172.21.15.157:0/2657115457 >> v2:172.21.15.131:6800/760045032 conn(0x7f32e406d990 0x7f32e406fe50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f32f400b5c0 tx=0x7f32f40037d0 comp rx=0 tx=0).stop 2024-02-05T07:49:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.520+0000 7f3301c31700 1 -- 172.21.15.157:0/2657115457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc105110 msgr2=0x7f32fc1978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:49:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.520+0000 7f3301c31700 1 --2- 172.21.15.157:0/2657115457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc105110 0x7f32fc1978c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32f000c9b0 tx=0x7f32f000ccc0 comp rx=0 tx=0).stop 2024-02-05T07:49:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.520+0000 7f3301c31700 1 -- 172.21.15.157:0/2657115457 shutdown_connections 2024-02-05T07:49:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.520+0000 7f3301c31700 1 --2- 172.21.15.157:0/2657115457 >> v2:172.21.15.131:6800/760045032 conn(0x7f32e406d990 0x7f32e406fe50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.520+0000 7f3301c31700 1 --2- 172.21.15.157:0/2657115457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32fc107a40 0x7f32fc19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.520+0000 7f3301c31700 1 --2- 172.21.15.157:0/2657115457 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f32fc1027e0 0x7f32fc197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.520+0000 7f3301c31700 1 --2- 172.21.15.157:0/2657115457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc105110 0x7f32fc1978c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:49:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.520+0000 7f3301c31700 1 -- 172.21.15.157:0/2657115457 >> 172.21.15.157:0/2657115457 conn(0x7f32fc0fc1f0 msgr2=0x7f32fc107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:49:59.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.520+0000 7f3301c31700 1 -- 172.21.15.157:0/2657115457 shutdown_connections 2024-02-05T07:49:59.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:49:59.520+0000 7f3301c31700 1 -- 172.21.15.157:0/2657115457 wait complete. 2024-02-05T07:50:00.384 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:50:00 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug Processing osd blocklist entries for this node 2024-02-05T07:50:00.421 DEBUG:teuthology.orchestra.run.smithi131:node-exporter.a> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@node-exporter.a.service 2024-02-05T07:50:00.424 DEBUG:teuthology.orchestra.run.smithi157:node-exporter.b> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@node-exporter.b.service 2024-02-05T07:50:00.426 INFO:tasks.cephadm:Adding alertmanager.a on smithi131 2024-02-05T07:50:00.426 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch apply alertmanager '1;smithi131=a' 2024-02-05T07:50:00.679 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:50:00 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug Reading the configuration object to update local LIO configuration 2024-02-05T07:50:00.679 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:50:00 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug Configuration does not have an entry for this host(smithi157.front.sepia.ceph.com) - nothing to define to LIO 2024-02-05T07:50:00.679 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:50:00 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2024-02-05T07:50:00.679 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:00 smithi157 ceph-mon[33345]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi131=a;smithi157=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:50:00.680 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:00 smithi157 ceph-mon[33345]: Saving service node-exporter spec with placement smithi131=a;smithi157=b;count:2 2024-02-05T07:50:00.680 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:00 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:00.680 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:00 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:00.680 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:00 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:00.680 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:00 smithi157 ceph-mon[33345]: overall HEALTH_OK 2024-02-05T07:50:00.680 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:00 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:00.680 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:00 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:00.680 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:00 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:50:00.680 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.157:0/103296048' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-05T07:50:00.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[41654]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi131=a;smithi157=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:50:00.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[41654]: Saving service node-exporter spec with placement smithi131=a;smithi157=b;count:2 2024-02-05T07:50:00.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:00.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:00.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:00.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[41654]: overall HEALTH_OK 2024-02-05T07:50:00.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:00.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:00.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:50:00.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.157:0/103296048' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-05T07:50:00.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[29507]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi131=a;smithi157=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:50:00.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[29507]: Saving service node-exporter spec with placement smithi131=a;smithi157=b;count:2 2024-02-05T07:50:00.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:00.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:00.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:00.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[29507]: overall HEALTH_OK 2024-02-05T07:50:00.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:00.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:00.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:50:00.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.157:0/103296048' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-05T07:50:01.052 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:50:01.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:01 smithi157 ceph-mon[33345]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-05T07:50:01.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:01 smithi157 ceph-mon[33345]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 4.7 KiB/s wr, 155 op/s 2024-02-05T07:50:01.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:01 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:01.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:01 smithi157 ceph-mon[33345]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-05T07:50:02.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:01 smithi131 ceph-mon[29507]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-05T07:50:02.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:01 smithi131 ceph-mon[29507]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 4.7 KiB/s wr, 155 op/s 2024-02-05T07:50:02.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:01 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:02.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:01 smithi131 ceph-mon[29507]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-05T07:50:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:01 smithi131 ceph-mon[41654]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-05T07:50:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:01 smithi131 ceph-mon[41654]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 4.7 KiB/s wr, 155 op/s 2024-02-05T07:50:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:01 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:01 smithi131 ceph-mon[41654]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-05T07:50:02.789 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.787+0000 7fc30c844700 1 -- 172.21.15.157:0/2057363988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3081051b0 msgr2=0x7fc3081075a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:02.789 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.787+0000 7fc30c844700 1 --2- 172.21.15.157:0/2057363988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3081051b0 0x7fc3081075a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009990 tx=0x7fc2fc009ca0 comp rx=0 tx=0).stop 2024-02-05T07:50:02.789 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.787+0000 7fc30c844700 1 -- 172.21.15.157:0/2057363988 shutdown_connections 2024-02-05T07:50:02.789 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.787+0000 7fc30c844700 1 --2- 172.21.15.157:0/2057363988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc308107ae0 0x7fc308109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:02.789 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.787+0000 7fc30c844700 1 --2- 172.21.15.157:0/2057363988 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc308102880 0x7fc308104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:02.789 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.787+0000 7fc30c844700 1 --2- 172.21.15.157:0/2057363988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3081051b0 0x7fc3081075a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:02.789 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.787+0000 7fc30c844700 1 -- 172.21.15.157:0/2057363988 >> 172.21.15.157:0/2057363988 conn(0x7fc3080fc2b0 msgr2=0x7fc3080fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:02.789 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.787+0000 7fc30c844700 1 -- 172.21.15.157:0/2057363988 shutdown_connections 2024-02-05T07:50:02.790 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.787+0000 7fc30c844700 1 -- 172.21.15.157:0/2057363988 wait complete. 2024-02-05T07:50:02.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc30c844700 1 Processor -- start 2024-02-05T07:50:02.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc30c844700 1 -- start start 2024-02-05T07:50:02.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc30c844700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc308102880 0x7fc308197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:02.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc30c844700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3081051b0 0x7fc308197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:02.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc30c844700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc308107ae0 0x7fc30819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:02.791 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc30c844700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30819f0e0 con 0x7fc3081051b0 2024-02-05T07:50:02.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc30c844700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30819f220 con 0x7fc308107ae0 2024-02-05T07:50:02.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc30c844700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30819f3b0 con 0x7fc308102880 2024-02-05T07:50:02.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc306d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc308107ae0 0x7fc30819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:02.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc30659c700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc308102880 0x7fc308197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:02.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc305d9b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3081051b0 0x7fc308197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:02.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc30659c700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc308102880 0x7fc308197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.157:35386/0 (socket says 172.21.15.157:35386) 2024-02-05T07:50:02.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc30659c700 1 -- 172.21.15.157:0/3679798993 learned_addr learned my addr 172.21.15.157:0/3679798993 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:50:02.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc306d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc308107ae0 0x7fc30819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45220/0 (socket says 172.21.15.157:45220) 2024-02-05T07:50:02.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc306d9d700 1 -- 172.21.15.157:0/3679798993 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc308102880 msgr2=0x7fc308197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:02.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc306d9d700 1 --2- 172.21.15.157:0/3679798993 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc308102880 0x7fc308197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:02.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc306d9d700 1 -- 172.21.15.157:0/3679798993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3081051b0 msgr2=0x7fc308197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:02.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc306d9d700 1 --2- 172.21.15.157:0/3679798993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3081051b0 0x7fc308197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:02.792 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.788+0000 7fc306d9d700 1 -- 172.21.15.157:0/3679798993 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc0095f0 con 0x7fc308107ae0 2024-02-05T07:50:02.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.789+0000 7fc306d9d700 1 --2- 172.21.15.157:0/3679798993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc308107ae0 0x7fc30819eb10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc30000bdf0 tx=0x7fc300009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:02.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.789+0000 7fc2f77fe700 1 -- 172.21.15.157:0/3679798993 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30000cb30 con 0x7fc308107ae0 2024-02-05T07:50:02.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.789+0000 7fc2f77fe700 1 -- 172.21.15.157:0/3679798993 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc30000cc90 con 0x7fc308107ae0 2024-02-05T07:50:02.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.789+0000 7fc30c844700 1 -- 172.21.15.157:0/3679798993 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30819f690 con 0x7fc308107ae0 2024-02-05T07:50:02.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.789+0000 7fc2f77fe700 1 -- 172.21.15.157:0/3679798993 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3000125a0 con 0x7fc308107ae0 2024-02-05T07:50:02.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.789+0000 7fc30c844700 1 -- 172.21.15.157:0/3679798993 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30819fbc0 con 0x7fc308107ae0 2024-02-05T07:50:02.793 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.790+0000 7fc30c844700 1 -- 172.21.15.157:0/3679798993 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3080fde30 con 0x7fc308107ae0 2024-02-05T07:50:02.794 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.790+0000 7fc2f77fe700 1 -- 172.21.15.157:0/3679798993 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 91423+0+0 (secure 0 0 0) 0x7fc300014440 con 0x7fc308107ae0 2024-02-05T07:50:02.794 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.790+0000 7fc2f77fe700 1 --2- 172.21.15.157:0/3679798993 >> v2:172.21.15.131:6800/760045032 conn(0x7fc2f006da20 0x7fc2f006fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:02.794 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.791+0000 7fc2f77fe700 1 -- 172.21.15.157:0/3679798993 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5942+0+0 (secure 0 0 0) 0x7fc300006e40 con 0x7fc308107ae0 2024-02-05T07:50:02.794 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.791+0000 7fc30659c700 1 --2- 172.21.15.157:0/3679798993 >> v2:172.21.15.131:6800/760045032 conn(0x7fc2f006da20 0x7fc2f006fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:02.794 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.791+0000 7fc30659c700 1 --2- 172.21.15.157:0/3679798993 >> v2:172.21.15.131:6800/760045032 conn(0x7fc2f006da20 0x7fc2f006fee0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009ad0 tx=0x7fc2f8009290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:02.797 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:02.795+0000 7fc2f77fe700 1 -- 172.21.15.157:0/3679798993 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc300056520 con 0x7fc308107ae0 2024-02-05T07:50:03.009 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:03.007+0000 7fc30c844700 1 -- 172.21.15.157:0/3679798993 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi131=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fc30810d0c0 con 0x7fc2f006da20 2024-02-05T07:50:03.016 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:03.014+0000 7fc2f77fe700 1 -- 172.21.15.157:0/3679798993 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fc30810d0c0 con 0x7fc2f006da20 2024-02-05T07:50:03.016 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled alertmanager update... 2024-02-05T07:50:03.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:03.016+0000 7fc30c844700 1 -- 172.21.15.157:0/3679798993 >> v2:172.21.15.131:6800/760045032 conn(0x7fc2f006da20 msgr2=0x7fc2f006fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:03.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:03.016+0000 7fc30c844700 1 --2- 172.21.15.157:0/3679798993 >> v2:172.21.15.131:6800/760045032 conn(0x7fc2f006da20 0x7fc2f006fee0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009ad0 tx=0x7fc2f8009290 comp rx=0 tx=0).stop 2024-02-05T07:50:03.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:03.016+0000 7fc30c844700 1 -- 172.21.15.157:0/3679798993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc308107ae0 msgr2=0x7fc30819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:03.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:03.016+0000 7fc30c844700 1 --2- 172.21.15.157:0/3679798993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc308107ae0 0x7fc30819eb10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc30000bdf0 tx=0x7fc300009ed0 comp rx=0 tx=0).stop 2024-02-05T07:50:03.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:03.016+0000 7fc30c844700 1 -- 172.21.15.157:0/3679798993 shutdown_connections 2024-02-05T07:50:03.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:03.016+0000 7fc30c844700 1 --2- 172.21.15.157:0/3679798993 >> v2:172.21.15.131:6800/760045032 conn(0x7fc2f006da20 0x7fc2f006fee0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:03.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:03.016+0000 7fc30c844700 1 --2- 172.21.15.157:0/3679798993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc308107ae0 0x7fc30819eb10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:03.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:03.016+0000 7fc30c844700 1 --2- 172.21.15.157:0/3679798993 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc308102880 0x7fc308197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:03.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:03.016+0000 7fc30c844700 1 --2- 172.21.15.157:0/3679798993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3081051b0 0x7fc308197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:03.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:03.016+0000 7fc30c844700 1 -- 172.21.15.157:0/3679798993 >> 172.21.15.157:0/3679798993 conn(0x7fc3080fc2b0 msgr2=0x7fc3081034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:03.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:03.016+0000 7fc30c844700 1 -- 172.21.15.157:0/3679798993 shutdown_connections 2024-02-05T07:50:03.019 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:03.016+0000 7fc30c844700 1 -- 172.21.15.157:0/3679798993 wait complete. 2024-02-05T07:50:03.691 DEBUG:teuthology.orchestra.run.smithi131:alertmanager.a> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@alertmanager.a.service 2024-02-05T07:50:03.694 INFO:tasks.cephadm:Adding grafana.a on smithi157 2024-02-05T07:50:03.694 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph orch apply grafana '1;smithi157=a' 2024-02-05T07:50:03.959 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:50:03.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:03 smithi157 ceph-mon[33345]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s rd, 4.2 KiB/s wr, 136 op/s 2024-02-05T07:50:03.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:03 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:04.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:03 smithi131 ceph-mon[41654]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s rd, 4.2 KiB/s wr, 136 op/s 2024-02-05T07:50:04.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:03 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:04.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:03 smithi131 ceph-mon[29507]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s rd, 4.2 KiB/s wr, 136 op/s 2024-02-05T07:50:04.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:03 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:04.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:04 smithi157 ceph-mon[33345]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi131=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:50:04.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:04 smithi157 ceph-mon[33345]: Saving service alertmanager spec with placement smithi131=a;count:1 2024-02-05T07:50:05.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:04 smithi131 ceph-mon[41654]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi131=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:50:05.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:04 smithi131 ceph-mon[41654]: Saving service alertmanager spec with placement smithi131=a;count:1 2024-02-05T07:50:05.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:04 smithi131 ceph-mon[29507]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi131=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:50:05.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:04 smithi131 ceph-mon[29507]: Saving service alertmanager spec with placement smithi131=a;count:1 2024-02-05T07:50:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:05 smithi157 ceph-mon[33345]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.4 KiB/s wr, 111 op/s 2024-02-05T07:50:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:50:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:50:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:05 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:05 smithi131 ceph-mon[41654]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.4 KiB/s wr, 111 op/s 2024-02-05T07:50:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:50:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:50:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:05 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:06.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:05 smithi131 ceph-mon[29507]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.4 KiB/s wr, 111 op/s 2024-02-05T07:50:06.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:06.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:50:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:50:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:05 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:06.464 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.462+0000 7f18d8db1700 1 -- 172.21.15.157:0/2431041063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d4100c50 msgr2=0x7f18d4101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:06.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.462+0000 7f18d8db1700 1 --2- 172.21.15.157:0/2431041063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d4100c50 0x7f18d4101070 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f18c80099c0 tx=0x7f18c8009cd0 comp rx=0 tx=0).stop 2024-02-05T07:50:06.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.462+0000 7f18d8db1700 1 -- 172.21.15.157:0/2431041063 shutdown_connections 2024-02-05T07:50:06.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.462+0000 7f18d8db1700 1 --2- 172.21.15.157:0/2431041063 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f18d41027f0 0x7f18d41090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:06.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.463+0000 7f18d8db1700 1 --2- 172.21.15.157:0/2431041063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d4100c50 0x7f18d4101070 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:06.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.463+0000 7f18d8db1700 1 --2- 172.21.15.157:0/2431041063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d4101e50 0x7f18d41022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:06.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.463+0000 7f18d8db1700 1 -- 172.21.15.157:0/2431041063 >> 172.21.15.157:0/2431041063 conn(0x7f18d40fc210 msgr2=0x7f18d40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:06.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.463+0000 7f18d8db1700 1 -- 172.21.15.157:0/2431041063 shutdown_connections 2024-02-05T07:50:06.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.463+0000 7f18d8db1700 1 -- 172.21.15.157:0/2431041063 wait complete. 2024-02-05T07:50:06.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.463+0000 7f18d8db1700 1 Processor -- start 2024-02-05T07:50:06.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.463+0000 7f18d8db1700 1 -- start start 2024-02-05T07:50:06.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.463+0000 7f18d8db1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d4100c50 0x7f18d406b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:06.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.463+0000 7f18d8db1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d4101e50 0x7f18d419c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:06.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.463+0000 7f18d8db1700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f18d41027f0 0x7f18d419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:06.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.463+0000 7f18d8db1700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d419ef20 con 0x7f18d4101e50 2024-02-05T07:50:06.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.463+0000 7f18d8db1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d419f090 con 0x7f18d4100c50 2024-02-05T07:50:06.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18d8db1700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d419f250 con 0x7f18d41027f0 2024-02-05T07:50:06.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18d259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d4100c50 0x7f18d406b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:06.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18d259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d4100c50 0x7f18d406b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45238/0 (socket says 172.21.15.157:45238) 2024-02-05T07:50:06.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18d259c700 1 -- 172.21.15.157:0/3409744297 learned_addr learned my addr 172.21.15.157:0/3409744297 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:50:06.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18d2d9d700 1 --2- 172.21.15.157:0/3409744297 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f18d41027f0 0x7f18d419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:06.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18d1d9b700 1 --2- 172.21.15.157:0/3409744297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d4101e50 0x7f18d419c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:06.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18d259c700 1 -- 172.21.15.157:0/3409744297 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f18d41027f0 msgr2=0x7f18d419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:06.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18d259c700 1 --2- 172.21.15.157:0/3409744297 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f18d41027f0 0x7f18d419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:06.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18d259c700 1 -- 172.21.15.157:0/3409744297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d4101e50 msgr2=0x7f18d419c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:06.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18d259c700 1 --2- 172.21.15.157:0/3409744297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d4101e50 0x7f18d419c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:06.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18d259c700 1 -- 172.21.15.157:0/3409744297 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c80095f0 con 0x7f18d4100c50 2024-02-05T07:50:06.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18d259c700 1 --2- 172.21.15.157:0/3409744297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d4100c50 0x7f18d406b000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f18c8006010 tx=0x7f18c8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:06.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18c37fe700 1 -- 172.21.15.157:0/3409744297 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c801d070 con 0x7f18d4100c50 2024-02-05T07:50:06.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18d8db1700 1 -- 172.21.15.157:0/3409744297 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d419f4d0 con 0x7f18d4100c50 2024-02-05T07:50:06.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18c37fe700 1 -- 172.21.15.157:0/3409744297 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f18c8004c90 con 0x7f18d4100c50 2024-02-05T07:50:06.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18c37fe700 1 -- 172.21.15.157:0/3409744297 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8005290 con 0x7f18d4100c50 2024-02-05T07:50:06.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.464+0000 7f18d8db1700 1 -- 172.21.15.157:0/3409744297 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d419f940 con 0x7f18d4100c50 2024-02-05T07:50:06.468 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.465+0000 7f18d8db1700 1 -- 172.21.15.157:0/3409744297 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d4107830 con 0x7f18d4100c50 2024-02-05T07:50:06.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.466+0000 7f18c37fe700 1 -- 172.21.15.157:0/3409744297 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 91423+0+0 (secure 0 0 0) 0x7f18c80053f0 con 0x7f18d4100c50 2024-02-05T07:50:06.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.466+0000 7f18c37fe700 1 --2- 172.21.15.157:0/3409744297 >> v2:172.21.15.131:6800/760045032 conn(0x7f18bc06da20 0x7f18bc06fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:06.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.466+0000 7f18c37fe700 1 -- 172.21.15.157:0/3409744297 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5942+0+0 (secure 0 0 0) 0x7f18c808e160 con 0x7f18d4100c50 2024-02-05T07:50:06.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.466+0000 7f18d1d9b700 1 --2- 172.21.15.157:0/3409744297 >> v2:172.21.15.131:6800/760045032 conn(0x7f18bc06da20 0x7f18bc06fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:06.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.467+0000 7f18d1d9b700 1 --2- 172.21.15.157:0/3409744297 >> v2:172.21.15.131:6800/760045032 conn(0x7f18bc06da20 0x7f18bc06fee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f18c4005950 tx=0x7f18c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:06.472 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.471+0000 7f18c37fe700 1 -- 172.21.15.157:0/3409744297 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f18c8057a90 con 0x7f18d4100c50 2024-02-05T07:50:06.724 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.723+0000 7f18d8db1700 1 -- 172.21.15.157:0/3409744297 --> v2:172.21.15.131:6800/760045032 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi157=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f18d410be30 con 0x7f18bc06da20 2024-02-05T07:50:06.732 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.730+0000 7f18c37fe700 1 -- 172.21.15.157:0/3409744297 <== mgr.14152 v2:172.21.15.131:6800/760045032 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f18d410be30 con 0x7f18bc06da20 2024-02-05T07:50:06.732 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled grafana update... 2024-02-05T07:50:06.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.733+0000 7f18d8db1700 1 -- 172.21.15.157:0/3409744297 >> v2:172.21.15.131:6800/760045032 conn(0x7f18bc06da20 msgr2=0x7f18bc06fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:06.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.733+0000 7f18d8db1700 1 --2- 172.21.15.157:0/3409744297 >> v2:172.21.15.131:6800/760045032 conn(0x7f18bc06da20 0x7f18bc06fee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f18c4005950 tx=0x7f18c400b410 comp rx=0 tx=0).stop 2024-02-05T07:50:06.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.733+0000 7f18d8db1700 1 -- 172.21.15.157:0/3409744297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d4100c50 msgr2=0x7f18d406b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:06.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.733+0000 7f18d8db1700 1 --2- 172.21.15.157:0/3409744297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d4100c50 0x7f18d406b000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f18c8006010 tx=0x7f18c8004730 comp rx=0 tx=0).stop 2024-02-05T07:50:06.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.733+0000 7f18d8db1700 1 -- 172.21.15.157:0/3409744297 shutdown_connections 2024-02-05T07:50:06.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.733+0000 7f18d8db1700 1 --2- 172.21.15.157:0/3409744297 >> v2:172.21.15.131:6800/760045032 conn(0x7f18bc06da20 0x7f18bc06fee0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:06.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.733+0000 7f18d8db1700 1 --2- 172.21.15.157:0/3409744297 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f18d41027f0 0x7f18d419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:06.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.733+0000 7f18d8db1700 1 --2- 172.21.15.157:0/3409744297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d4100c50 0x7f18d406b000 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:06.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.733+0000 7f18d8db1700 1 --2- 172.21.15.157:0/3409744297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18d4101e50 0x7f18d419c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:06.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.733+0000 7f18d8db1700 1 -- 172.21.15.157:0/3409744297 >> 172.21.15.157:0/3409744297 conn(0x7f18d40fc210 msgr2=0x7f18d40fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:06.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.733+0000 7f18d8db1700 1 -- 172.21.15.157:0/3409744297 shutdown_connections 2024-02-05T07:50:06.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:06.733+0000 7f18d8db1700 1 -- 172.21.15.157:0/3409744297 wait complete. 2024-02-05T07:50:06.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:06 smithi157 ceph-mon[33345]: Deploying daemon prometheus.a on smithi157 2024-02-05T07:50:07.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:06 smithi131 ceph-mon[41654]: Deploying daemon prometheus.a on smithi157 2024-02-05T07:50:07.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:06 smithi131 ceph-mon[29507]: Deploying daemon prometheus.a on smithi157 2024-02-05T07:50:07.366 DEBUG:teuthology.orchestra.run.smithi157:grafana.a> sudo journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@grafana.a.service 2024-02-05T07:50:07.368 INFO:tasks.cephadm:Setting up client nodes... 2024-02-05T07:50:07.368 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-05T07:50:07.620 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:08.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:07 smithi131 ceph-mon[41654]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 35 KiB/s rd, 2.6 KiB/s wr, 84 op/s 2024-02-05T07:50:08.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:07 smithi131 ceph-mon[41654]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi157=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:50:08.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:07 smithi131 ceph-mon[41654]: Saving service grafana spec with placement smithi157=a;count:1 2024-02-05T07:50:08.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:07 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:08.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:07 smithi131 ceph-mon[29507]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 35 KiB/s rd, 2.6 KiB/s wr, 84 op/s 2024-02-05T07:50:08.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:07 smithi131 ceph-mon[29507]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi157=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:50:08.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:07 smithi131 ceph-mon[29507]: Saving service grafana spec with placement smithi157=a;count:1 2024-02-05T07:50:08.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:07 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:08.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:07 smithi157 ceph-mon[33345]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 35 KiB/s rd, 2.6 KiB/s wr, 84 op/s 2024-02-05T07:50:08.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:07 smithi157 ceph-mon[33345]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi157=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:50:08.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:07 smithi157 ceph-mon[33345]: Saving service grafana spec with placement smithi157=a;count:1 2024-02-05T07:50:08.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:07 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:09.146 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:08 smithi131 ceph-mon[41654]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 756 B/s wr, 25 op/s 2024-02-05T07:50:09.146 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:08 smithi131 ceph-mon[29507]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 756 B/s wr, 25 op/s 2024-02-05T07:50:09.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:08 smithi157 ceph-mon[33345]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 756 B/s wr, 25 op/s 2024-02-05T07:50:09.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 -- 172.21.15.131:0/3324685169 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc704107a50 msgr2=0x7fc704109ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:09.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 --2- 172.21.15.131:0/3324685169 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc704107a50 0x7fc704109ec0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00b0a0 tx=0x7fc6fc00b3b0 comp rx=0 tx=0).stop 2024-02-05T07:50:09.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 -- 172.21.15.131:0/3324685169 shutdown_connections 2024-02-05T07:50:09.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 --2- 172.21.15.131:0/3324685169 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc704107a50 0x7fc704109ec0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:09.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 --2- 172.21.15.131:0/3324685169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7041027f0 0x7fc704104be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:09.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 --2- 172.21.15.131:0/3324685169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc704105120 0x7fc704107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:09.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 -- 172.21.15.131:0/3324685169 >> 172.21.15.131:0/3324685169 conn(0x7fc7040fc220 msgr2=0x7fc7040fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:09.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 -- 172.21.15.131:0/3324685169 shutdown_connections 2024-02-05T07:50:09.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 -- 172.21.15.131:0/3324685169 wait complete. 2024-02-05T07:50:09.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 Processor -- start 2024-02-05T07:50:09.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 -- start start 2024-02-05T07:50:09.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc7041027f0 0x7fc704197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:09.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc704105120 0x7fc704197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:09.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc704107a50 0x7fc70419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:09.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70419ef60 con 0x7fc704105120 2024-02-05T07:50:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70419f0a0 con 0x7fc704107a50 2024-02-05T07:50:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.387+0000 7fc70b106700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70419f230 con 0x7fc7041027f0 2024-02-05T07:50:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc708ea2700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc7041027f0 0x7fc704197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc703fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc704105120 0x7fc704197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc708ea2700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc7041027f0 0x7fc704197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:55244/0 (socket says 172.21.15.131:55244) 2024-02-05T07:50:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc703fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc704105120 0x7fc704197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56488/0 (socket says 172.21.15.131:56488) 2024-02-05T07:50:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc708ea2700 1 -- 172.21.15.131:0/64545290 learned_addr learned my addr 172.21.15.131:0/64545290 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc7096a3700 1 --2- 172.21.15.131:0/64545290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc704107a50 0x7fc70419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc708ea2700 1 -- 172.21.15.131:0/64545290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc704107a50 msgr2=0x7fc70419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc708ea2700 1 --2- 172.21.15.131:0/64545290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc704107a50 0x7fc70419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc708ea2700 1 -- 172.21.15.131:0/64545290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc704105120 msgr2=0x7fc704197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc708ea2700 1 --2- 172.21.15.131:0/64545290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc704105120 0x7fc704197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc708ea2700 1 -- 172.21.15.131:0/64545290 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f0009520 con 0x7fc7041027f0 2024-02-05T07:50:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc703fff700 1 --2- 172.21.15.131:0/64545290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc704105120 0x7fc704197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:50:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc708ea2700 1 --2- 172.21.15.131:0/64545290 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc7041027f0 0x7fc704197420 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800c950 tx=0x7fc6f800cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc701ffb700 1 -- 172.21.15.131:0/64545290 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f80078d0 con 0x7fc7041027f0 2024-02-05T07:50:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc701ffb700 1 -- 172.21.15.131:0/64545290 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc6f800f450 con 0x7fc7041027f0 2024-02-05T07:50:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc70b106700 1 -- 172.21.15.131:0/64545290 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6fc009d00 con 0x7fc7041027f0 2024-02-05T07:50:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.388+0000 7fc701ffb700 1 -- 172.21.15.131:0/64545290 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8018660 con 0x7fc7041027f0 2024-02-05T07:50:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.389+0000 7fc70b106700 1 -- 172.21.15.131:0/64545290 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70419f880 con 0x7fc7041027f0 2024-02-05T07:50:09.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.389+0000 7fc70b106700 1 -- 172.21.15.131:0/64545290 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc704050230 con 0x7fc7041027f0 2024-02-05T07:50:09.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.390+0000 7fc701ffb700 1 -- 172.21.15.131:0/64545290 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 16) v1 ==== 91423+0+0 (secure 0 0 0) 0x7fc6f8007a30 con 0x7fc7041027f0 2024-02-05T07:50:09.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.390+0000 7fc701ffb700 1 --2- 172.21.15.131:0/64545290 >> v2:172.21.15.131:6800/760045032 conn(0x7fc6f406dac0 0x7fc6f406ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:09.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.390+0000 7fc703fff700 1 --2- 172.21.15.131:0/64545290 >> v2:172.21.15.131:6800/760045032 conn(0x7fc6f406dac0 0x7fc6f406ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:09.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.390+0000 7fc701ffb700 1 -- 172.21.15.131:0/64545290 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5942+0+0 (secure 0 0 0) 0x7fc6f808d0f0 con 0x7fc7041027f0 2024-02-05T07:50:09.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.391+0000 7fc703fff700 1 --2- 172.21.15.131:0/64545290 >> v2:172.21.15.131:6800/760045032 conn(0x7fc6f406dac0 0x7fc6f406ff80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0009fd0 tx=0x7fc6f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:09.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.395+0000 7fc701ffb700 1 -- 172.21.15.131:0/64545290 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc6f8056a20 con 0x7fc7041027f0 2024-02-05T07:50:09.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.670+0000 7fc70b106700 1 -- 172.21.15.131:0/64545290 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fc70410cd50 con 0x7fc7041027f0 2024-02-05T07:50:09.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.674+0000 7fc701ffb700 1 -- 172.21.15.131:0/64545290 <== mon.2 v2:172.21.15.131:3301/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7fc70410cd50 con 0x7fc7041027f0 2024-02-05T07:50:09.676 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-02-05T07:50:09.676 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAxk8BlX80eKBAARBymr0LZvX3x6zLFhUo/AQ== 2024-02-05T07:50:09.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.676+0000 7fc70b106700 1 -- 172.21.15.131:0/64545290 >> v2:172.21.15.131:6800/760045032 conn(0x7fc6f406dac0 msgr2=0x7fc6f406ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:09.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.676+0000 7fc70b106700 1 --2- 172.21.15.131:0/64545290 >> v2:172.21.15.131:6800/760045032 conn(0x7fc6f406dac0 0x7fc6f406ff80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0009fd0 tx=0x7fc6f0009260 comp rx=0 tx=0).stop 2024-02-05T07:50:09.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.676+0000 7fc70b106700 1 -- 172.21.15.131:0/64545290 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc7041027f0 msgr2=0x7fc704197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:09.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.676+0000 7fc70b106700 1 --2- 172.21.15.131:0/64545290 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc7041027f0 0x7fc704197420 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800c950 tx=0x7fc6f800cc60 comp rx=0 tx=0).stop 2024-02-05T07:50:09.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.676+0000 7fc70b106700 1 -- 172.21.15.131:0/64545290 shutdown_connections 2024-02-05T07:50:09.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.676+0000 7fc70b106700 1 --2- 172.21.15.131:0/64545290 >> v2:172.21.15.131:6800/760045032 conn(0x7fc6f406dac0 0x7fc6f406ff80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:09.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.676+0000 7fc70b106700 1 --2- 172.21.15.131:0/64545290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc704107a50 0x7fc70419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:09.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.676+0000 7fc70b106700 1 --2- 172.21.15.131:0/64545290 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc7041027f0 0x7fc704197420 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:09.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.676+0000 7fc70b106700 1 --2- 172.21.15.131:0/64545290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc704105120 0x7fc704197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:09.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.677+0000 7fc70b106700 1 -- 172.21.15.131:0/64545290 >> 172.21.15.131:0/64545290 conn(0x7fc7040fc220 msgr2=0x7fc704107e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:09.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.677+0000 7fc70b106700 1 -- 172.21.15.131:0/64545290 shutdown_connections 2024-02-05T07:50:09.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:09.677+0000 7fc70b106700 1 -- 172.21.15.131:0/64545290 wait complete. 2024-02-05T07:50:09.966 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/64545290' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:50:09.966 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:50:09.966 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T07:50:09.966 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/64545290' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:50:09.966 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:50:09.967 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T07:50:10.139 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/64545290' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:50:10.139 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:50:10.139 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T07:50:10.311 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:50:10.311 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-05T07:50:10.311 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-05T07:50:10.353 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-05T07:50:10.471 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:50:10 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:50:10.588 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:50:10.965 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:10 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:10.965 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:10 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-02-05T07:50:10.965 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:10 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-02-05T07:50:10.965 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:10 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2024-02-05T07:50:10.965 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:10 smithi157 ceph-mon[33345]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 597 B/s wr, 19 op/s 2024-02-05T07:50:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:10 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:10 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-02-05T07:50:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:10 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-02-05T07:50:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:10 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2024-02-05T07:50:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:10 smithi131 ceph-mon[41654]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 597 B/s wr, 19 op/s 2024-02-05T07:50:11.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:10 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-02-05T07:50:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-02-05T07:50:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:10 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]: dispatch 2024-02-05T07:50:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:10 smithi131 ceph-mon[29507]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 597 B/s wr, 19 op/s 2024-02-05T07:50:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:11 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-02-05T07:50:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:11 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-02-05T07:50:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:11 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]': finished 2024-02-05T07:50:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:11 smithi157 ceph-mon[33345]: osdmap e72: 8 total, 8 up, 8 in 2024-02-05T07:50:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:11 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-02-05T07:50:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:11 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-02-05T07:50:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:11 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]': finished 2024-02-05T07:50:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:11 smithi131 ceph-mon[41654]: osdmap e72: 8 total, 8 up, 8 in 2024-02-05T07:50:12.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:11 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-02-05T07:50:12.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:11 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-02-05T07:50:12.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:11 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 5]}]': finished 2024-02-05T07:50:12.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:11 smithi131 ceph-mon[29507]: osdmap e72: 8 total, 8 up, 8 in 2024-02-05T07:50:13.126 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:12 smithi157 ceph-mon[33345]: osdmap e73: 8 total, 8 up, 8 in 2024-02-05T07:50:13.127 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:12 smithi157 ceph-mon[33345]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 2 op/s 2024-02-05T07:50:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:12 smithi131 ceph-mon[41654]: osdmap e73: 8 total, 8 up, 8 in 2024-02-05T07:50:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:12 smithi131 ceph-mon[41654]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 2 op/s 2024-02-05T07:50:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:12 smithi131 ceph-mon[29507]: osdmap e73: 8 total, 8 up, 8 in 2024-02-05T07:50:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:12 smithi131 ceph-mon[29507]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 2 op/s 2024-02-05T07:50:14.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.261+0000 7f9205958700 1 -- 172.21.15.157:0/528059830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92001039f0 msgr2=0x7f920010a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:14.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.261+0000 7f9205958700 1 --2- 172.21.15.157:0/528059830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92001039f0 0x7f920010a2a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009990 tx=0x7f91f0009ca0 comp rx=0 tx=0).stop 2024-02-05T07:50:14.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.261+0000 7f9205958700 1 -- 172.21.15.157:0/528059830 shutdown_connections 2024-02-05T07:50:14.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.261+0000 7f9205958700 1 --2- 172.21.15.157:0/528059830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92001039f0 0x7f920010a2a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:14.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.261+0000 7f9205958700 1 --2- 172.21.15.157:0/528059830 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9200100070 0x7f9200100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:14.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.261+0000 7f9205958700 1 --2- 172.21.15.157:0/528059830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9200103050 0x7f92001034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:14.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.261+0000 7f9205958700 1 -- 172.21.15.157:0/528059830 >> 172.21.15.157:0/528059830 conn(0x7f92000fb9d0 msgr2=0x7f92000fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:14.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.261+0000 7f9205958700 1 -- 172.21.15.157:0/528059830 shutdown_connections 2024-02-05T07:50:14.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.261+0000 7f9205958700 1 -- 172.21.15.157:0/528059830 wait complete. 2024-02-05T07:50:14.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.261+0000 7f9205958700 1 Processor -- start 2024-02-05T07:50:14.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.261+0000 7f9205958700 1 -- start start 2024-02-05T07:50:14.302 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f9205958700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9200100070 0x7f92001974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:14.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f9205958700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9200103050 0x7f9200197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:14.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f9205958700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92001039f0 0x7f920019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:14.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f9205958700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920019f010 con 0x7f9200100070 2024-02-05T07:50:14.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f9205958700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920019f150 con 0x7f92001039f0 2024-02-05T07:50:14.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f9205958700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920019f2e0 con 0x7f9200103050 2024-02-05T07:50:14.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f91ff7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92001039f0 0x7f920019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:14.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f91ff7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92001039f0 0x7f920019ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35530/0 (socket says 172.21.15.157:35530) 2024-02-05T07:50:14.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f91ff7fe700 1 -- 172.21.15.157:0/1527557682 learned_addr learned my addr 172.21.15.157:0/1527557682 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-02-05T07:50:14.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f91fe7fc700 1 --2- 172.21.15.157:0/1527557682 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9200103050 0x7f9200197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:14.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f91feffd700 1 --2- 172.21.15.157:0/1527557682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9200100070 0x7f92001974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:14.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f91ff7fe700 1 -- 172.21.15.157:0/1527557682 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9200103050 msgr2=0x7f9200197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:14.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f91ff7fe700 1 --2- 172.21.15.157:0/1527557682 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9200103050 0x7f9200197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:14.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f91ff7fe700 1 -- 172.21.15.157:0/1527557682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9200100070 msgr2=0x7f92001974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:14.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f91ff7fe700 1 --2- 172.21.15.157:0/1527557682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9200100070 0x7f92001974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:14.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f91ff7fe700 1 -- 172.21.15.157:0/1527557682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f00095f0 con 0x7f92001039f0 2024-02-05T07:50:14.303 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f91ff7fe700 1 --2- 172.21.15.157:0/1527557682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92001039f0 0x7f920019ea40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91f0003840 tx=0x7f91f0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:14.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f9204956700 1 -- 172.21.15.157:0/1527557682 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f001d070 con 0x7f92001039f0 2024-02-05T07:50:14.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f9205958700 1 -- 172.21.15.157:0/1527557682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920019f560 con 0x7f92001039f0 2024-02-05T07:50:14.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f9204956700 1 -- 172.21.15.157:0/1527557682 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f91f0004050 con 0x7f92001039f0 2024-02-05T07:50:14.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f9204956700 1 -- 172.21.15.157:0/1527557682 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0022d50 con 0x7f92001039f0 2024-02-05T07:50:14.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.262+0000 7f9205958700 1 -- 172.21.15.157:0/1527557682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920019fa50 con 0x7f92001039f0 2024-02-05T07:50:14.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.263+0000 7f9205958700 1 -- 172.21.15.157:0/1527557682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9200108a30 con 0x7f92001039f0 2024-02-05T07:50:14.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.264+0000 7f9204956700 1 -- 172.21.15.157:0/1527557682 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 91423+0+0 (secure 0 0 0) 0x7f91f0003ba0 con 0x7f92001039f0 2024-02-05T07:50:14.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.264+0000 7f9204956700 1 --2- 172.21.15.157:0/1527557682 >> v2:172.21.15.131:6800/760045032 conn(0x7f91e806dac0 0x7f91e806ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:14.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.264+0000 7f9204956700 1 -- 172.21.15.157:0/1527557682 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f91f008e8f0 con 0x7f92001039f0 2024-02-05T07:50:14.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.264+0000 7f91feffd700 1 --2- 172.21.15.157:0/1527557682 >> v2:172.21.15.131:6800/760045032 conn(0x7f91e806dac0 0x7f91e806ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:14.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.265+0000 7f91feffd700 1 --2- 172.21.15.157:0/1527557682 >> v2:172.21.15.131:6800/760045032 conn(0x7f91e806dac0 0x7f91e806ff80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f920006a280 tx=0x7f91f800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:14.304 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.269+0000 7f9204956700 1 -- 172.21.15.157:0/1527557682 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f91f00581e0 con 0x7f92001039f0 2024-02-05T07:50:14.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.546+0000 7f9205958700 1 -- 172.21.15.157:0/1527557682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f920010cf00 con 0x7f92001039f0 2024-02-05T07:50:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.551+0000 7f9204956700 1 -- 172.21.15.157:0/1527557682 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f91f005b8a0 con 0x7f92001039f0 2024-02-05T07:50:14.553 INFO:teuthology.orchestra.run.smithi157.stdout:[client.1] 2024-02-05T07:50:14.553 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQA2k8BlnSG6IBAAtwZfmnsKKRCrdIMcUebH6w== 2024-02-05T07:50:14.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.553+0000 7f9205958700 1 -- 172.21.15.157:0/1527557682 >> v2:172.21.15.131:6800/760045032 conn(0x7f91e806dac0 msgr2=0x7f91e806ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:14.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.553+0000 7f9205958700 1 --2- 172.21.15.157:0/1527557682 >> v2:172.21.15.131:6800/760045032 conn(0x7f91e806dac0 0x7f91e806ff80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f920006a280 tx=0x7f91f800d040 comp rx=0 tx=0).stop 2024-02-05T07:50:14.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.553+0000 7f9205958700 1 -- 172.21.15.157:0/1527557682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92001039f0 msgr2=0x7f920019ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:14.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.553+0000 7f9205958700 1 --2- 172.21.15.157:0/1527557682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92001039f0 0x7f920019ea40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91f0003840 tx=0x7f91f0004730 comp rx=0 tx=0).stop 2024-02-05T07:50:14.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.553+0000 7f9205958700 1 -- 172.21.15.157:0/1527557682 shutdown_connections 2024-02-05T07:50:14.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.553+0000 7f9205958700 1 --2- 172.21.15.157:0/1527557682 >> v2:172.21.15.131:6800/760045032 conn(0x7f91e806dac0 0x7f91e806ff80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:14.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.553+0000 7f9205958700 1 --2- 172.21.15.157:0/1527557682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9200100070 0x7f92001974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:14.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.553+0000 7f9205958700 1 --2- 172.21.15.157:0/1527557682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92001039f0 0x7f920019ea40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:14.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.553+0000 7f9205958700 1 --2- 172.21.15.157:0/1527557682 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9200103050 0x7f9200197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:14.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.553+0000 7f9205958700 1 -- 172.21.15.157:0/1527557682 >> 172.21.15.157:0/1527557682 conn(0x7f92000fb9d0 msgr2=0x7f92000fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:14.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.553+0000 7f9205958700 1 -- 172.21.15.157:0/1527557682 shutdown_connections 2024-02-05T07:50:14.556 INFO:teuthology.orchestra.run.smithi157.stderr:2024-02-05T07:50:14.553+0000 7f9205958700 1 -- 172.21.15.157:0/1527557682 wait complete. 2024-02-05T07:50:14.807 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.157:0/1527557682' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:50:14.807 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:50:14.807 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T07:50:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.157:0/1527557682' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:50:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:50:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T07:50:15.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.157:0/1527557682' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:50:15.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:50:15.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T07:50:15.845 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:15 smithi157 ceph-mon[33345]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 0 B/s, 1 objects/s recovering 2024-02-05T07:50:15.876 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-02-05T07:50:15.876 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-05T07:50:15.876 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-05T07:50:15.921 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-05T07:50:15.922 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-05T07:50:15.922 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph mgr dump --format=json 2024-02-05T07:50:15.951 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:15 smithi131 ceph-mon[41654]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 0 B/s, 1 objects/s recovering 2024-02-05T07:50:15.951 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:15 smithi131 ceph-mon[29507]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 0 B/s, 1 objects/s recovering 2024-02-05T07:50:16.172 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:17.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.933+0000 7fe59ce4f700 1 -- 172.21.15.131:0/2715245736 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe598102730 msgr2=0x7fe59810ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:17.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.933+0000 7fe59ce4f700 1 --2- 172.21.15.131:0/2715245736 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe598102730 0x7fe59810ebd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe59000b0a0 tx=0x7fe59000b3b0 comp rx=0 tx=0).stop 2024-02-05T07:50:17.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.933+0000 7fe59ce4f700 1 -- 172.21.15.131:0/2715245736 shutdown_connections 2024-02-05T07:50:17.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.933+0000 7fe59ce4f700 1 --2- 172.21.15.131:0/2715245736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe598107e20 0x7fe598108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:17.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.933+0000 7fe59ce4f700 1 --2- 172.21.15.131:0/2715245736 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe598102730 0x7fe59810ebd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:17.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.933+0000 7fe59ce4f700 1 --2- 172.21.15.131:0/2715245736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe598101d70 0x7fe5981021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:17.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.933+0000 7fe59ce4f700 1 -- 172.21.15.131:0/2715245736 >> 172.21.15.131:0/2715245736 conn(0x7fe598078e50 msgr2=0x7fe598079260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:17.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.933+0000 7fe59ce4f700 1 -- 172.21.15.131:0/2715245736 shutdown_connections 2024-02-05T07:50:17.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.933+0000 7fe59ce4f700 1 -- 172.21.15.131:0/2715245736 wait complete. 2024-02-05T07:50:17.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.934+0000 7fe59ce4f700 1 Processor -- start 2024-02-05T07:50:17.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.934+0000 7fe59ce4f700 1 -- start start 2024-02-05T07:50:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.934+0000 7fe59ce4f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe598101d70 0x7fe59819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.934+0000 7fe59ce4f700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe598102730 0x7fe59819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.934+0000 7fe59ce4f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe598107e20 0x7fe598197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.934+0000 7fe59ce4f700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe598197ed0 con 0x7fe598101d70 2024-02-05T07:50:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.934+0000 7fe59ce4f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe598198040 con 0x7fe598107e20 2024-02-05T07:50:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.934+0000 7fe59ce4f700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe598198200 con 0x7fe598102730 2024-02-05T07:50:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe595d9b700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe598102730 0x7fe59819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe59659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe598101d70 0x7fe59819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe59659c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe598101d70 0x7fe59819d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56512/0 (socket says 172.21.15.131:56512) 2024-02-05T07:50:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe59659c700 1 -- 172.21.15.131:0/3983346597 learned_addr learned my addr 172.21.15.131:0/3983346597 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe595d9b700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe598102730 0x7fe59819db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:55266/0 (socket says 172.21.15.131:55266) 2024-02-05T07:50:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe596d9d700 1 --2- 172.21.15.131:0/3983346597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe598107e20 0x7fe598197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe595d9b700 1 -- 172.21.15.131:0/3983346597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe598107e20 msgr2=0x7fe598197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe595d9b700 1 --2- 172.21.15.131:0/3983346597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe598107e20 0x7fe598197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe595d9b700 1 -- 172.21.15.131:0/3983346597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe598101d70 msgr2=0x7fe59819d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:17.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe595d9b700 1 --2- 172.21.15.131:0/3983346597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe598101d70 0x7fe59819d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:17.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe595d9b700 1 -- 172.21.15.131:0/3983346597 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58c009520 con 0x7fe598102730 2024-02-05T07:50:17.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe59659c700 1 --2- 172.21.15.131:0/3983346597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe598101d70 0x7fe59819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:50:17.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe595d9b700 1 --2- 172.21.15.131:0/3983346597 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe598102730 0x7fe59819db90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe58800d950 tx=0x7fe58800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:17.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe5877fe700 1 -- 172.21.15.131:0/3983346597 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588009930 con 0x7fe598102730 2024-02-05T07:50:17.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe5877fe700 1 -- 172.21.15.131:0/3983346597 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fe588010460 con 0x7fe598102730 2024-02-05T07:50:17.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe59ce4f700 1 -- 172.21.15.131:0/3983346597 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe590009d00 con 0x7fe598102730 2024-02-05T07:50:17.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe5877fe700 1 -- 172.21.15.131:0/3983346597 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58800b5d0 con 0x7fe598102730 2024-02-05T07:50:17.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.935+0000 7fe59ce4f700 1 -- 172.21.15.131:0/3983346597 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5981a44b0 con 0x7fe598102730 2024-02-05T07:50:17.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.936+0000 7fe59ce4f700 1 -- 172.21.15.131:0/3983346597 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe598103460 con 0x7fe598102730 2024-02-05T07:50:17.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.937+0000 7fe5877fe700 1 -- 172.21.15.131:0/3983346597 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 16) v1 ==== 91423+0+0 (secure 0 0 0) 0x7fe588009a90 con 0x7fe598102730 2024-02-05T07:50:17.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.937+0000 7fe5877fe700 1 --2- 172.21.15.131:0/3983346597 >> v2:172.21.15.131:6800/760045032 conn(0x7fe58006da20 0x7fe58006fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:17.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.937+0000 7fe5877fe700 1 -- 172.21.15.131:0/3983346597 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fe58808ce60 con 0x7fe598102730 2024-02-05T07:50:17.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.937+0000 7fe59659c700 1 --2- 172.21.15.131:0/3983346597 >> v2:172.21.15.131:6800/760045032 conn(0x7fe58006da20 0x7fe58006fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:17.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.937+0000 7fe59659c700 1 --2- 172.21.15.131:0/3983346597 >> v2:172.21.15.131:6800/760045032 conn(0x7fe58006da20 0x7fe58006fee0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009fd0 tx=0x7fe58c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:17.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:17.942+0000 7fe5877fe700 1 -- 172.21.15.131:0/3983346597 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe588056750 con 0x7fe598102730 2024-02-05T07:50:17.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:17 smithi131 ceph-mon[41654]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-02-05T07:50:17.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:17 smithi131 ceph-mon[29507]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-02-05T07:50:17.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:17 smithi157 ceph-mon[33345]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-02-05T07:50:18.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:18.190+0000 7fe59ce4f700 1 -- 172.21.15.131:0/3983346597 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fe598198d00 con 0x7fe598102730 2024-02-05T07:50:18.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:18.193+0000 7fe5877fe700 1 -- 172.21.15.131:0/3983346597 <== mon.2 v2:172.21.15.131:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) v1 ==== 74+0+174806 (secure 0 0 0) 0x7fe588059e10 con 0x7fe598102730 2024-02-05T07:50:18.197 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:50:18.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:18.196+0000 7fe59ce4f700 1 -- 172.21.15.131:0/3983346597 >> v2:172.21.15.131:6800/760045032 conn(0x7fe58006da20 msgr2=0x7fe58006fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:18.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:18.196+0000 7fe59ce4f700 1 --2- 172.21.15.131:0/3983346597 >> v2:172.21.15.131:6800/760045032 conn(0x7fe58006da20 0x7fe58006fee0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009fd0 tx=0x7fe58c009260 comp rx=0 tx=0).stop 2024-02-05T07:50:18.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:18.196+0000 7fe59ce4f700 1 -- 172.21.15.131:0/3983346597 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe598102730 msgr2=0x7fe59819db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:18.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:18.196+0000 7fe59ce4f700 1 --2- 172.21.15.131:0/3983346597 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe598102730 0x7fe59819db90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe58800d950 tx=0x7fe58800dc60 comp rx=0 tx=0).stop 2024-02-05T07:50:18.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:18.196+0000 7fe59ce4f700 1 -- 172.21.15.131:0/3983346597 shutdown_connections 2024-02-05T07:50:18.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:18.196+0000 7fe59ce4f700 1 --2- 172.21.15.131:0/3983346597 >> v2:172.21.15.131:6800/760045032 conn(0x7fe58006da20 0x7fe58006fee0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:18.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:18.196+0000 7fe59ce4f700 1 --2- 172.21.15.131:0/3983346597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe598101d70 0x7fe59819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:18.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:18.196+0000 7fe59ce4f700 1 --2- 172.21.15.131:0/3983346597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe598107e20 0x7fe598197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:18.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:18.196+0000 7fe59ce4f700 1 --2- 172.21.15.131:0/3983346597 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe598102730 0x7fe59819db90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:18.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:18.196+0000 7fe59ce4f700 1 -- 172.21.15.131:0/3983346597 >> 172.21.15.131:0/3983346597 conn(0x7fe598078e50 msgr2=0x7fe598105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:18.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:18.196+0000 7fe59ce4f700 1 -- 172.21.15.131:0/3983346597 shutdown_connections 2024-02-05T07:50:18.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:18.196+0000 7fe59ce4f700 1 -- 172.21.15.131:0/3983346597 wait complete. 2024-02-05T07:50:18.471 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 systemd[1]: Starting Ceph prometheus.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T07:50:18.753 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 podman[65128]: 2024-02-05 07:50:18.401255058 +0000 UTC m=+0.019664873 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-05T07:50:18.753 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 podman[65128]: 2024-02-05 07:50:18.526108572 +0000 UTC m=+0.144518387 container create 6e137e49b25a93ee24668bfa603aed1e89769ee73e81a15ef43442d8989983c4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:50:19.014 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":16,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":760045032}]},"active_addr":"172.21.15.131:6800/760045032","active_change":"2024-02-05T07:45:10.598898+0000","active_mgr_features":4540701272860131327,"available":true,"standbys":[{"gid":24106,"name":"x","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.131:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":1652019838}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":1797722746}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":1533300721}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":3989773858}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":2313739489}]}]} 2024-02-05T07:50:19.018 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-05T07:50:19.018 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-05T07:50:19.018 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-05T07:50:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3983346597' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-05T07:50:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-mon[33345]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 13 B/s, 1 objects/s recovering 2024-02-05T07:50:19.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 podman[65128]: 2024-02-05 07:50:18.753062003 +0000 UTC m=+0.371471818 container init 6e137e49b25a93ee24668bfa603aed1e89769ee73e81a15ef43442d8989983c4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:50:19.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 podman[65128]: 2024-02-05 07:50:18.758836226 +0000 UTC m=+0.377246042 container start 6e137e49b25a93ee24668bfa603aed1e89769ee73e81a15ef43442d8989983c4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:50:19.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.782Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-05T07:50:19.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.782Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-05T07:50:19.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.782Z caller=main.go:570 level=info host_details="(Linux 5.14.0-416.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Jan 31 22:52:48 UTC 2024 x86_64 smithi157 (none))" 2024-02-05T07:50:19.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.782Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-05T07:50:19.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.782Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-05T07:50:19.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.784Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-05T07:50:19.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.785Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-05T07:50:19.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.785Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-05T07:50:19.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.785Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-05T07:50:19.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.786Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-05T07:50:19.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.787Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.444µs 2024-02-05T07:50:19.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.787Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-05T07:50:19.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.787Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-05T07:50:19.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.787Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=16.08µs wal_replay_duration=379.386µs wbl_replay_duration=399ns total_replay_duration=413.068µs 2024-02-05T07:50:19.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.788Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-05T07:50:19.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.788Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-05T07:50:19.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.788Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-05T07:50:19.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.805Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=16.996253ms db_storage=895ns remote_storage=1.339µs web_handler=271ns query_engine=652ns scrape=450.522µs scrape_sd=78.717µs notify=763ns notify_sd=1.258µs rules=16.289575ms tracing=7.251µs 2024-02-05T07:50:19.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.805Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-05T07:50:19.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:50:18.805Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-05T07:50:19.224 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 bash[65128]: 6e137e49b25a93ee24668bfa603aed1e89769ee73e81a15ef43442d8989983c4 2024-02-05T07:50:19.224 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:50:18 smithi157 systemd[1]: Started Ceph prometheus.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T07:50:19.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3983346597' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-05T07:50:19.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:18 smithi131 ceph-mon[41654]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 13 B/s, 1 objects/s recovering 2024-02-05T07:50:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3983346597' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-05T07:50:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:18 smithi131 ceph-mon[29507]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 13 B/s, 1 objects/s recovering 2024-02-05T07:50:19.295 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:20.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:19 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:20.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:19 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:20.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:19 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:20.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:19 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-05T07:50:20.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:19 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ignoring --setuser ceph since I am not root 2024-02-05T07:50:20.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:19 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ignoring --setgroup ceph since I am not root 2024-02-05T07:50:20.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:19 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:20.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:19 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:20.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:19 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:20.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:19 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-05T07:50:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:19 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:19 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:19 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' 2024-02-05T07:50:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:19 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-05T07:50:20.221 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: ignoring --setuser ceph since I am not root 2024-02-05T07:50:20.222 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: ignoring --setgroup ceph since I am not root 2024-02-05T07:50:20.222 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:19 smithi157 ceph-mgr[35456]: -- 172.21.15.157:0/3342952388 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560a99e79520 con 0x560a99e4ec00 2024-02-05T07:50:20.743 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:20 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:20.464+0000 7f36ee208200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T07:50:20.743 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:20 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:20.559+0000 7f36ee208200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T07:50:20.897 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:20 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:20.470+0000 7f1f9bba1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T07:50:20.897 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:20 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:20.563+0000 7f1f9bba1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T07:50:21.013 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:20 smithi131 ceph-mon[41654]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-05T07:50:21.014 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:20 smithi131 ceph-mon[41654]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-05T07:50:21.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:20 smithi131 ceph-mon[29507]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-05T07:50:21.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:20 smithi131 ceph-mon[29507]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-05T07:50:21.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.116+0000 7f6e074bf700 1 -- 172.21.15.131:0/2853146528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e00100c20 msgr2=0x7f6e00101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:21.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.116+0000 7f6e074bf700 1 --2- 172.21.15.131:0/2853146528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e00100c20 0x7f6e00101080 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6df00098f0 tx=0x7f6df0009c00 comp rx=0 tx=0).stop 2024-02-05T07:50:21.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.116+0000 7f6e074bf700 1 -- 172.21.15.131:0/2853146528 shutdown_connections 2024-02-05T07:50:21.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.116+0000 7f6e074bf700 1 --2- 172.21.15.131:0/2853146528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e001015c0 0x7f6e0010daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:21.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.116+0000 7f6e074bf700 1 --2- 172.21.15.131:0/2853146528 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f6e00106c60 0x7f6e00107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:21.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.116+0000 7f6e074bf700 1 --2- 172.21.15.131:0/2853146528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e00100c20 0x7f6e00101080 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:21.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.116+0000 7f6e074bf700 1 -- 172.21.15.131:0/2853146528 >> 172.21.15.131:0/2853146528 conn(0x7f6e00077b50 msgr2=0x7f6e00077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:21.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.116+0000 7f6e074bf700 1 -- 172.21.15.131:0/2853146528 shutdown_connections 2024-02-05T07:50:21.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.116+0000 7f6e074bf700 1 -- 172.21.15.131:0/2853146528 wait complete. 2024-02-05T07:50:21.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.117+0000 7f6e074bf700 1 Processor -- start 2024-02-05T07:50:21.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.117+0000 7f6e074bf700 1 -- start start 2024-02-05T07:50:21.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.117+0000 7f6e074bf700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f6e00100c20 0x7f6e00198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:21.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.117+0000 7f6e074bf700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e001015c0 0x7f6e00198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:21.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.117+0000 7f6e074bf700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e00106c60 0x7f6e001a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:21.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.117+0000 7f6e074bf700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e00199650 con 0x7f6e001015c0 2024-02-05T07:50:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.117+0000 7f6e074bf700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e001a3dd0 con 0x7f6e00106c60 2024-02-05T07:50:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.117+0000 7f6e074bf700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e001a3f40 con 0x7f6e00100c20 2024-02-05T07:50:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.117+0000 7f6e04a5a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e001015c0 0x7f6e00198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.117+0000 7f6e0525b700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f6e00100c20 0x7f6e00198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.117+0000 7f6e04a5a700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e001015c0 0x7f6e00198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50034/0 (socket says 172.21.15.131:50034) 2024-02-05T07:50:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.117+0000 7f6e0525b700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f6e00100c20 0x7f6e00198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:46068/0 (socket says 172.21.15.131:46068) 2024-02-05T07:50:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.117+0000 7f6e04a5a700 1 -- 172.21.15.131:0/2202133798 learned_addr learned my addr 172.21.15.131:0/2202133798 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.117+0000 7f6e05a5c700 1 --2- 172.21.15.131:0/2202133798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e00106c60 0x7f6e001a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.118+0000 7f6e04a5a700 1 -- 172.21.15.131:0/2202133798 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f6e00100c20 msgr2=0x7f6e00198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.118+0000 7f6e04a5a700 1 --2- 172.21.15.131:0/2202133798 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f6e00100c20 0x7f6e00198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.118+0000 7f6e04a5a700 1 -- 172.21.15.131:0/2202133798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e00106c60 msgr2=0x7f6e001a3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.118+0000 7f6e04a5a700 1 --2- 172.21.15.131:0/2202133798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e00106c60 0x7f6e001a3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.118+0000 7f6e04a5a700 1 -- 172.21.15.131:0/2202133798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df0009520 con 0x7f6e001015c0 2024-02-05T07:50:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.118+0000 7f6e0525b700 1 --2- 172.21.15.131:0/2202133798 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f6e00100c20 0x7f6e00198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:50:21.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.118+0000 7f6e04a5a700 1 --2- 172.21.15.131:0/2202133798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e001015c0 0x7f6e00198d50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6df000f9e0 tx=0x7f6df000fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:21.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.118+0000 7f6df67fc700 1 -- 172.21.15.131:0/2202133798 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df001c070 con 0x7f6e001015c0 2024-02-05T07:50:21.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.118+0000 7f6e074bf700 1 -- 172.21.15.131:0/2202133798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e001a41c0 con 0x7f6e001015c0 2024-02-05T07:50:21.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.118+0000 7f6e074bf700 1 -- 172.21.15.131:0/2202133798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e001a45d0 con 0x7f6e001015c0 2024-02-05T07:50:21.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.118+0000 7f6df67fc700 1 -- 172.21.15.131:0/2202133798 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f6df000bb10 con 0x7f6e001015c0 2024-02-05T07:50:21.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.118+0000 7f6df67fc700 1 -- 172.21.15.131:0/2202133798 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0004930 con 0x7f6e001015c0 2024-02-05T07:50:21.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.119+0000 7f6e074bf700 1 -- 172.21.15.131:0/2202133798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de4005120 con 0x7f6e001015c0 2024-02-05T07:50:21.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.120+0000 7f6df67fc700 1 -- 172.21.15.131:0/2202133798 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 91437+0+0 (secure 0 0 0) 0x7f6df0020070 con 0x7f6e001015c0 2024-02-05T07:50:21.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.120+0000 7f6df67fc700 1 --2- 172.21.15.131:0/2202133798 >> v2:172.21.15.131:6800/760045032 conn(0x7f6dec06da70 0x7f6dec06ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:21.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.120+0000 7f6e0525b700 1 -- 172.21.15.131:0/2202133798 >> v2:172.21.15.131:6800/760045032 conn(0x7f6dec06da70 msgr2=0x7f6dec06ff30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/760045032 2024-02-05T07:50:21.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.120+0000 7f6df67fc700 1 -- 172.21.15.131:0/2202133798 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f6df008e070 con 0x7f6e001015c0 2024-02-05T07:50:21.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.120+0000 7f6e0525b700 1 --2- 172.21.15.131:0/2202133798 >> v2:172.21.15.131:6800/760045032 conn(0x7f6dec06da70 0x7f6dec06ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-05T07:50:21.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.129+0000 7f6df67fc700 1 -- 172.21.15.131:0/2202133798 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6df0057a80 con 0x7f6e001015c0 2024-02-05T07:50:21.208 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:20 smithi157 ceph-mon[33345]: from='mgr.14152 172.21.15.131:0/1647720302' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-05T07:50:21.208 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:20 smithi157 ceph-mon[33345]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-05T07:50:21.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:21 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:21.230+0000 7f36ee208200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T07:50:21.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.321+0000 7f6e0525b700 1 -- 172.21.15.131:0/2202133798 >> v2:172.21.15.131:6800/760045032 conn(0x7f6dec06da70 msgr2=0x7f6dec06ff30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/760045032 2024-02-05T07:50:21.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.321+0000 7f6e0525b700 1 --2- 172.21.15.131:0/2202133798 >> v2:172.21.15.131:6800/760045032 conn(0x7f6dec06da70 0x7f6dec06ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-05T07:50:21.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.332+0000 7f6e074bf700 1 -- 172.21.15.131:0/2202133798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6de40056a0 con 0x7f6e001015c0 2024-02-05T07:50:21.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.334+0000 7f6df67fc700 1 -- 172.21.15.131:0/2202133798 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+20719 (secure 0 0 0) 0x7f6df005b140 con 0x7f6e001015c0 2024-02-05T07:50:21.336 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:50:21.336 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":73,"fsid":"4018fbe8-c3fa-11ee-95b6-87774f69a715","created":"2024-02-05T07:44:07.660911+0000","modified":"2024-02-05T07:50:11.849252+0000","last_up_change":"2024-02-05T07:49:33.716478+0000","last_in_change":"2024-02-05T07:49:15.544169+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-05T07:47:28.930141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-02-05T07:49:45.334857+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-05T07:49:45.866499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-05T07:49:47.822233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-05T07:49:49.691217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-05T07:49:51.758187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f89dd964-ec08-4f4b-9e8f-91bea0156d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6801","nonce":1614402360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1614402360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1614402360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6803","nonce":1614402360}]},"public_addr":"172.21.15.131:6801/1614402360","cluster_addr":"172.21.15.131:6802/1614402360","heartbeat_back_addr":"172.21.15.131:6804/1614402360","heartbeat_front_addr":"172.21.15.131:6803/1614402360","state":["exists","up"]},{"osd":1,"uuid":"b6068c48-3bc2-4728-9924-66bafeff3703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6805","nonce":1256644882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1256644882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1256644882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6807","nonce":1256644882}]},"public_addr":"172.21.15.131:6805/1256644882","cluster_addr":"172.21.15.131:6806/1256644882","heartbeat_back_addr":"172.21.15.131:6808/1256644882","heartbeat_front_addr":"172.21.15.131:6807/1256644882","state":["exists","up"]},{"osd":2,"uuid":"8d0cfb98-091c-467e-abe5-abe3b8542323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6809","nonce":1463881636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1463881636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1463881636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6811","nonce":1463881636}]},"public_addr":"172.21.15.131:6809/1463881636","cluster_addr":"172.21.15.131:6810/1463881636","heartbeat_back_addr":"172.21.15.131:6812/1463881636","heartbeat_front_addr":"172.21.15.131:6811/1463881636","state":["exists","up"]},{"osd":3,"uuid":"3e262dfc-6d5a-4f3f-b605-6c92e6839f62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":3290626285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3290626285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3290626285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6815","nonce":3290626285}]},"public_addr":"172.21.15.131:6813/3290626285","cluster_addr":"172.21.15.131:6814/3290626285","heartbeat_back_addr":"172.21.15.131:6816/3290626285","heartbeat_front_addr":"172.21.15.131:6815/3290626285","state":["exists","up"]},{"osd":4,"uuid":"f6925a13-9a72-42bd-97b2-970f9e709dad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1828659015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":1828659015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6803","nonce":1828659015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1828659015}]},"public_addr":"172.21.15.157:6800/1828659015","cluster_addr":"172.21.15.157:6801/1828659015","heartbeat_back_addr":"172.21.15.157:6803/1828659015","heartbeat_front_addr":"172.21.15.157:6802/1828659015","state":["exists","up"]},{"osd":5,"uuid":"96c3644d-9b3a-4826-befe-0f6cc4af9baf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3589104149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":3589104149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":3589104149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3589104149}]},"public_addr":"172.21.15.157:6804/3589104149","cluster_addr":"172.21.15.157:6805/3589104149","heartbeat_back_addr":"172.21.15.157:6807/3589104149","heartbeat_front_addr":"172.21.15.157:6806/3589104149","state":["exists","up"]},{"osd":6,"uuid":"24119b57-55ed-4b51-b25e-7a6cea556af3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":204226792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":204226792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":204226792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":204226792}]},"public_addr":"172.21.15.157:6808/204226792","cluster_addr":"172.21.15.157:6809/204226792","heartbeat_back_addr":"172.21.15.157:6811/204226792","heartbeat_front_addr":"172.21.15.157:6810/204226792","state":["exists","up"]},{"osd":7,"uuid":"d15bc7ec-b99f-4130-b071-3493f4ac7b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1810544184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":1810544184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":1810544184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1810544184}]},"public_addr":"172.21.15.157:6812/1810544184","cluster_addr":"172.21.15.157:6813/1810544184","heartbeat_back_addr":"172.21.15.157:6815/1810544184","heartbeat_front_addr":"172.21.15.157:6814/1810544184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:46:34.040947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:46:59.284998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:47:25.195950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:47:51.338667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:48:14.496069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:48:39.323248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:49:05.597163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:49:31.741281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/1270355705":"2024-02-06T07:45:10.598786+0000","172.21.15.131:6800/2947584178":"2024-02-06T07:45:10.598786+0000","172.21.15.131:0/3554035799":"2024-02-06T07:44:42.598552+0000","172.21.15.131:0/3462494148":"2024-02-06T07:44:42.598552+0000","172.21.15.131:6800/1416975616":"2024-02-06T07:44:42.598552+0000","172.21.15.131:0/219975829":"2024-02-06T07:44:42.598552+0000","172.21.15.131:0/1166974614":"2024-02-06T07:45:10.598786+0000","172.21.15.131:0/3288446474":"2024-02-06T07:45:10.598786+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-05T07:50:21.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.337+0000 7f6e074bf700 1 -- 172.21.15.131:0/2202133798 >> v2:172.21.15.131:6800/760045032 conn(0x7f6dec06da70 msgr2=0x7f6dec06ff30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-05T07:50:21.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.337+0000 7f6e074bf700 1 --2- 172.21.15.131:0/2202133798 >> v2:172.21.15.131:6800/760045032 conn(0x7f6dec06da70 0x7f6dec06ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:21.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.337+0000 7f6e074bf700 1 -- 172.21.15.131:0/2202133798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e001015c0 msgr2=0x7f6e00198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:21.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.337+0000 7f6e074bf700 1 --2- 172.21.15.131:0/2202133798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e001015c0 0x7f6e00198d50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6df000f9e0 tx=0x7f6df000fa10 comp rx=0 tx=0).stop 2024-02-05T07:50:21.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.337+0000 7f6e074bf700 1 -- 172.21.15.131:0/2202133798 shutdown_connections 2024-02-05T07:50:21.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.337+0000 7f6e074bf700 1 --2- 172.21.15.131:0/2202133798 >> v2:172.21.15.131:6800/760045032 conn(0x7f6dec06da70 0x7f6dec06ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:21.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.337+0000 7f6e074bf700 1 --2- 172.21.15.131:0/2202133798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e00106c60 0x7f6e001a3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:21.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.337+0000 7f6e074bf700 1 --2- 172.21.15.131:0/2202133798 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f6e00100c20 0x7f6e00198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:21.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.337+0000 7f6e074bf700 1 --2- 172.21.15.131:0/2202133798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e001015c0 0x7f6e00198d50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:21.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.337+0000 7f6e074bf700 1 -- 172.21.15.131:0/2202133798 >> 172.21.15.131:0/2202133798 conn(0x7f6e00077b50 msgr2=0x7f6e000ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:21.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.337+0000 7f6e074bf700 1 -- 172.21.15.131:0/2202133798 shutdown_connections 2024-02-05T07:50:21.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:21.337+0000 7f6e074bf700 1 -- 172.21.15.131:0/2202133798 wait complete. 2024-02-05T07:50:21.471 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:21 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:21.206+0000 7f1f9bba1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T07:50:21.471 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:21 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:21.300+0000 7f1f9bba1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T07:50:21.471 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:21 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:21.444+0000 7f1f9bba1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T07:50:21.577 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:21 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:21.325+0000 7f36ee208200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T07:50:21.577 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:21 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:21.479+0000 7f36ee208200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T07:50:21.891 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:21 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:21.543+0000 7f1f9bba1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T07:50:21.892 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:21 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:21.627+0000 7f1f9bba1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T07:50:21.892 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:21 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:21.890+0000 7f1f9bba1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T07:50:21.898 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:21 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:21.575+0000 7f36ee208200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T07:50:21.898 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:21 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:21.659+0000 7f36ee208200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T07:50:22.129 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-05T07:50:22.129 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-05T07:50:22.154 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2202133798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:50:22.155 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:21 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:21.928+0000 7f36ee208200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T07:50:22.155 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:22.038+0000 7f36ee208200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T07:50:22.155 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:22.125+0000 7f36ee208200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T07:50:22.155 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2202133798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:50:22.184 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2202133798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:50:22.184 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:22 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:22.001+0000 7f1f9bba1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T07:50:22.185 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:22 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:22.087+0000 7f1f9bba1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T07:50:22.185 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:22 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:22.183+0000 7f1f9bba1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T07:50:22.380 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:22.422 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:22.226+0000 7f36ee208200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T07:50:22.471 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:22 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:22.367+0000 7f1f9bba1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T07:50:22.686 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:22.421+0000 7f36ee208200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T07:50:23.543 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:23 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:23.408+0000 7f36ee208200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T07:50:23.721 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:23 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:23.373+0000 7f1f9bba1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T07:50:23.721 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:23 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:23.512+0000 7f1f9bba1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T07:50:23.888 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:23 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:23.542+0000 7f36ee208200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T07:50:24.136 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:23 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:23.850+0000 7f1f9bba1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T07:50:24.136 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:23 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:23.940+0000 7f1f9bba1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T07:50:24.136 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:24 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:24.134+0000 7f1f9bba1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:50:24.187 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:23 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:23.887+0000 7f36ee208200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T07:50:24.187 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:23 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:23.979+0000 7f36ee208200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T07:50:24.187 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:24 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:24.186+0000 7f36ee208200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:50:24.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.349+0000 7f22feb3c700 1 -- 172.21.15.131:0/611897357 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f22f8101860 msgr2=0x7f22f8105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.349+0000 7f22feb3c700 1 --2- 172.21.15.131:0/611897357 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f22f8101860 0x7f22f8105820 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f22e40098f0 tx=0x7f22e4009c00 comp rx=0 tx=0).stop 2024-02-05T07:50:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.349+0000 7f22feb3c700 1 -- 172.21.15.131:0/611897357 shutdown_connections 2024-02-05T07:50:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.349+0000 7f22feb3c700 1 --2- 172.21.15.131:0/611897357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22f8105f00 0x7f22f810dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.349+0000 7f22feb3c700 1 --2- 172.21.15.131:0/611897357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f8100eb0 0x7f22f8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.349+0000 7f22feb3c700 1 --2- 172.21.15.131:0/611897357 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f22f8101860 0x7f22f8105820 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.349+0000 7f22feb3c700 1 -- 172.21.15.131:0/611897357 >> 172.21.15.131:0/611897357 conn(0x7f22f8076c70 msgr2=0x7f22f8077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.349+0000 7f22feb3c700 1 -- 172.21.15.131:0/611897357 shutdown_connections 2024-02-05T07:50:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.349+0000 7f22feb3c700 1 -- 172.21.15.131:0/611897357 wait complete. 2024-02-05T07:50:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22feb3c700 1 Processor -- start 2024-02-05T07:50:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22feb3c700 1 -- start start 2024-02-05T07:50:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22feb3c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f8100eb0 0x7f22f810e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:24.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22feb3c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22f8101860 0x7f22f810e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22feb3c700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f22f8105f00 0x7f22f810ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22feb3c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f81a5df0 con 0x7f22f8101860 2024-02-05T07:50:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22feb3c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f81a5f60 con 0x7f22f8100eb0 2024-02-05T07:50:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22feb3c700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f81a6120 con 0x7f22f8105f00 2024-02-05T07:50:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22f7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22f8101860 0x7f22f810e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22f7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22f8101860 0x7f22f810e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50058/0 (socket says 172.21.15.131:50058) 2024-02-05T07:50:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22fd0d9700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f22f8105f00 0x7f22f810ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:24.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22f7fff700 1 -- 172.21.15.131:0/1687549286 learned_addr learned my addr 172.21.15.131:0/1687549286 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22fc8d8700 1 --2- 172.21.15.131:0/1687549286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f8100eb0 0x7f22f810e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22f7fff700 1 -- 172.21.15.131:0/1687549286 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f22f8105f00 msgr2=0x7f22f810ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22f7fff700 1 --2- 172.21.15.131:0/1687549286 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f22f8105f00 0x7f22f810ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22f7fff700 1 -- 172.21.15.131:0/1687549286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f8100eb0 msgr2=0x7f22f810e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22f7fff700 1 --2- 172.21.15.131:0/1687549286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f8100eb0 0x7f22f810e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22f7fff700 1 -- 172.21.15.131:0/1687549286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f0009d00 con 0x7f22f8101860 2024-02-05T07:50:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.350+0000 7f22fd0d9700 1 --2- 172.21.15.131:0/1687549286 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f22f8105f00 0x7f22f810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:50:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.351+0000 7f22f7fff700 1 --2- 172.21.15.131:0/1687549286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22f8101860 0x7f22f810e920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009230 tx=0x7f22e400f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.351+0000 7f22f5ffb700 1 -- 172.21.15.131:0/1687549286 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e401c070 con 0x7f22f8101860 2024-02-05T07:50:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.351+0000 7f22f5ffb700 1 -- 172.21.15.131:0/1687549286 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f22e400ba90 con 0x7f22f8101860 2024-02-05T07:50:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.351+0000 7f22feb3c700 1 -- 172.21.15.131:0/1687549286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e4009520 con 0x7f22f8101860 2024-02-05T07:50:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.351+0000 7f22f5ffb700 1 -- 172.21.15.131:0/1687549286 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e40038e0 con 0x7f22f8101860 2024-02-05T07:50:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.351+0000 7f22feb3c700 1 -- 172.21.15.131:0/1687549286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f81a6680 con 0x7f22f8101860 2024-02-05T07:50:24.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.352+0000 7f22feb3c700 1 -- 172.21.15.131:0/1687549286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f8109220 con 0x7f22f8101860 2024-02-05T07:50:24.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.353+0000 7f22f5ffb700 1 -- 172.21.15.131:0/1687549286 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 17) v1 ==== 91437+0+0 (secure 0 0 0) 0x7f22e4020030 con 0x7f22f8101860 2024-02-05T07:50:24.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.353+0000 7f22f5ffb700 1 --2- 172.21.15.131:0/1687549286 >> v2:172.21.15.131:6800/760045032 conn(0x7f22e806da70 0x7f22e806ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:24.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.353+0000 7f22fc8d8700 1 -- 172.21.15.131:0/1687549286 >> v2:172.21.15.131:6800/760045032 conn(0x7f22e806da70 msgr2=0x7f22e806ff30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/760045032 2024-02-05T07:50:24.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.353+0000 7f22f5ffb700 1 -- 172.21.15.131:0/1687549286 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f22e408e140 con 0x7f22f8101860 2024-02-05T07:50:24.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.353+0000 7f22fc8d8700 1 --2- 172.21.15.131:0/1687549286 >> v2:172.21.15.131:6800/760045032 conn(0x7f22e806da70 0x7f22e806ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-05T07:50:24.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.358+0000 7f22f5ffb700 1 -- 172.21.15.131:0/1687549286 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f22e4057ab0 con 0x7f22f8101860 2024-02-05T07:50:24.471 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:24 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:24.317+0000 7f1f9bba1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T07:50:24.517 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:24 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:24.381+0000 7f36ee208200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T07:50:24.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.554+0000 7f22fc8d8700 1 -- 172.21.15.131:0/1687549286 >> v2:172.21.15.131:6800/760045032 conn(0x7f22e806da70 msgr2=0x7f22e806ff30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.131:6800/760045032 2024-02-05T07:50:24.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.554+0000 7f22fc8d8700 1 --2- 172.21.15.131:0/1687549286 >> v2:172.21.15.131:6800/760045032 conn(0x7f22e806da70 0x7f22e806ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-05T07:50:24.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.557+0000 7f22feb3c700 1 -- 172.21.15.131:0/1687549286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22f810f910 con 0x7f22f8101860 2024-02-05T07:50:24.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.559+0000 7f22f5ffb700 1 -- 172.21.15.131:0/1687549286 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+20719 (secure 0 0 0) 0x7f22e405b170 con 0x7f22f8101860 2024-02-05T07:50:24.560 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:50:24.560 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":73,"fsid":"4018fbe8-c3fa-11ee-95b6-87774f69a715","created":"2024-02-05T07:44:07.660911+0000","modified":"2024-02-05T07:50:11.849252+0000","last_up_change":"2024-02-05T07:49:33.716478+0000","last_in_change":"2024-02-05T07:49:15.544169+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-05T07:47:28.930141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-02-05T07:49:45.334857+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-05T07:49:45.866499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-05T07:49:47.822233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-05T07:49:49.691217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-05T07:49:51.758187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f89dd964-ec08-4f4b-9e8f-91bea0156d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6801","nonce":1614402360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":1614402360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":1614402360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6803","nonce":1614402360}]},"public_addr":"172.21.15.131:6801/1614402360","cluster_addr":"172.21.15.131:6802/1614402360","heartbeat_back_addr":"172.21.15.131:6804/1614402360","heartbeat_front_addr":"172.21.15.131:6803/1614402360","state":["exists","up"]},{"osd":1,"uuid":"b6068c48-3bc2-4728-9924-66bafeff3703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6805","nonce":1256644882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":1256644882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1256644882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6807","nonce":1256644882}]},"public_addr":"172.21.15.131:6805/1256644882","cluster_addr":"172.21.15.131:6806/1256644882","heartbeat_back_addr":"172.21.15.131:6808/1256644882","heartbeat_front_addr":"172.21.15.131:6807/1256644882","state":["exists","up"]},{"osd":2,"uuid":"8d0cfb98-091c-467e-abe5-abe3b8542323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6809","nonce":1463881636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1463881636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1463881636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6811","nonce":1463881636}]},"public_addr":"172.21.15.131:6809/1463881636","cluster_addr":"172.21.15.131:6810/1463881636","heartbeat_back_addr":"172.21.15.131:6812/1463881636","heartbeat_front_addr":"172.21.15.131:6811/1463881636","state":["exists","up"]},{"osd":3,"uuid":"3e262dfc-6d5a-4f3f-b605-6c92e6839f62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":3290626285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3290626285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3290626285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6815","nonce":3290626285}]},"public_addr":"172.21.15.131:6813/3290626285","cluster_addr":"172.21.15.131:6814/3290626285","heartbeat_back_addr":"172.21.15.131:6816/3290626285","heartbeat_front_addr":"172.21.15.131:6815/3290626285","state":["exists","up"]},{"osd":4,"uuid":"f6925a13-9a72-42bd-97b2-970f9e709dad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1828659015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":1828659015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6803","nonce":1828659015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1828659015}]},"public_addr":"172.21.15.157:6800/1828659015","cluster_addr":"172.21.15.157:6801/1828659015","heartbeat_back_addr":"172.21.15.157:6803/1828659015","heartbeat_front_addr":"172.21.15.157:6802/1828659015","state":["exists","up"]},{"osd":5,"uuid":"96c3644d-9b3a-4826-befe-0f6cc4af9baf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":3589104149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6805","nonce":3589104149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":3589104149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":3589104149}]},"public_addr":"172.21.15.157:6804/3589104149","cluster_addr":"172.21.15.157:6805/3589104149","heartbeat_back_addr":"172.21.15.157:6807/3589104149","heartbeat_front_addr":"172.21.15.157:6806/3589104149","state":["exists","up"]},{"osd":6,"uuid":"24119b57-55ed-4b51-b25e-7a6cea556af3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":204226792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6809","nonce":204226792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":204226792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":204226792}]},"public_addr":"172.21.15.157:6808/204226792","cluster_addr":"172.21.15.157:6809/204226792","heartbeat_back_addr":"172.21.15.157:6811/204226792","heartbeat_front_addr":"172.21.15.157:6810/204226792","state":["exists","up"]},{"osd":7,"uuid":"d15bc7ec-b99f-4130-b071-3493f4ac7b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1810544184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6813","nonce":1810544184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6815","nonce":1810544184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1810544184}]},"public_addr":"172.21.15.157:6812/1810544184","cluster_addr":"172.21.15.157:6813/1810544184","heartbeat_back_addr":"172.21.15.157:6815/1810544184","heartbeat_front_addr":"172.21.15.157:6814/1810544184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:46:34.040947+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:46:59.284998+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:47:25.195950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:47:51.338667+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:48:14.496069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:48:39.323248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:49:05.597163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:49:31.741281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:0/1270355705":"2024-02-06T07:45:10.598786+0000","172.21.15.131:6800/2947584178":"2024-02-06T07:45:10.598786+0000","172.21.15.131:0/3554035799":"2024-02-06T07:44:42.598552+0000","172.21.15.131:0/3462494148":"2024-02-06T07:44:42.598552+0000","172.21.15.131:6800/1416975616":"2024-02-06T07:44:42.598552+0000","172.21.15.131:0/219975829":"2024-02-06T07:44:42.598552+0000","172.21.15.131:0/1166974614":"2024-02-06T07:45:10.598786+0000","172.21.15.131:0/3288446474":"2024-02-06T07:45:10.598786+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-05T07:50:24.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.561+0000 7f22feb3c700 1 -- 172.21.15.131:0/1687549286 >> v2:172.21.15.131:6800/760045032 conn(0x7f22e806da70 msgr2=0x7f22e806ff30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-05T07:50:24.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.561+0000 7f22feb3c700 1 --2- 172.21.15.131:0/1687549286 >> v2:172.21.15.131:6800/760045032 conn(0x7f22e806da70 0x7f22e806ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:24.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.561+0000 7f22feb3c700 1 -- 172.21.15.131:0/1687549286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22f8101860 msgr2=0x7f22f810e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:24.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.561+0000 7f22feb3c700 1 --2- 172.21.15.131:0/1687549286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22f8101860 0x7f22f810e920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009230 tx=0x7f22e400f9e0 comp rx=0 tx=0).stop 2024-02-05T07:50:24.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.561+0000 7f22feb3c700 1 -- 172.21.15.131:0/1687549286 shutdown_connections 2024-02-05T07:50:24.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.561+0000 7f22feb3c700 1 --2- 172.21.15.131:0/1687549286 >> v2:172.21.15.131:6800/760045032 conn(0x7f22e806da70 0x7f22e806ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:24.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.561+0000 7f22feb3c700 1 --2- 172.21.15.131:0/1687549286 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f22f8105f00 0x7f22f810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:24.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.561+0000 7f22feb3c700 1 --2- 172.21.15.131:0/1687549286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f8100eb0 0x7f22f810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:24.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.561+0000 7f22feb3c700 1 --2- 172.21.15.131:0/1687549286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22f8101860 0x7f22f810e920 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.561+0000 7f22feb3c700 1 -- 172.21.15.131:0/1687549286 >> 172.21.15.131:0/1687549286 conn(0x7f22f8076c70 msgr2=0x7f22f81070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.561+0000 7f22feb3c700 1 -- 172.21.15.131:0/1687549286 shutdown_connections 2024-02-05T07:50:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:24.561+0000 7f22feb3c700 1 -- 172.21.15.131:0/1687549286 wait complete. 2024-02-05T07:50:24.824 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:24 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1687549286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:50:24.825 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1687549286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:50:24.825 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:24 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:24.681+0000 7f36ee208200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:50:24.862 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:24 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1687549286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:50:24.862 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:24 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:24.600+0000 7f1f9bba1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:50:25.126 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:24 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:24.948+0000 7f36ee208200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T07:50:25.155 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-05T07:50:25.155 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-05T07:50:25.156 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-05T07:50:25.156 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-05T07:50:25.157 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-05T07:50:25.157 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-05T07:50:25.157 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-05T07:50:25.158 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-05T07:50:25.221 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:24 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:24.860+0000 7f1f9bba1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T07:50:25.221 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:25 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:25.094+0000 7f1f9bba1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T07:50:25.518 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:25 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:25.194+0000 7f36ee208200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T07:50:25.798 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:25.798 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:25.800 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:25.803 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:25.803 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:25.803 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:25.806 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:25.806 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:26.221 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:25 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:25.892+0000 7f1f9bba1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T07:50:26.518 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:26 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:26.292+0000 7f36ee208200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T07:50:26.721 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:26 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:26.464+0000 7f1f9bba1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T07:50:26.721 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:26 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:26.563+0000 7f1f9bba1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T07:50:27.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:26 smithi131 ceph-mon[41654]: Standby manager daemon x restarted 2024-02-05T07:50:27.132 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:26 smithi131 ceph-mon[41654]: Standby manager daemon x started 2024-02-05T07:50:27.132 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:26 smithi131 ceph-mon[41654]: from='mgr.? 172.21.15.157:0/1324886962' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T07:50:27.132 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:26 smithi131 ceph-mon[41654]: from='mgr.? 172.21.15.157:0/1324886962' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T07:50:27.132 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:26 smithi131 ceph-mon[41654]: from='mgr.? 172.21.15.157:0/1324886962' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T07:50:27.132 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:26 smithi131 ceph-mon[41654]: from='mgr.? 172.21.15.157:0/1324886962' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T07:50:27.132 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:26 smithi131 ceph-mon[29507]: Standby manager daemon x restarted 2024-02-05T07:50:27.132 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:26 smithi131 ceph-mon[29507]: Standby manager daemon x started 2024-02-05T07:50:27.132 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:26 smithi131 ceph-mon[29507]: from='mgr.? 172.21.15.157:0/1324886962' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T07:50:27.133 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:26 smithi131 ceph-mon[29507]: from='mgr.? 172.21.15.157:0/1324886962' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T07:50:27.133 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:26 smithi131 ceph-mon[29507]: from='mgr.? 172.21.15.157:0/1324886962' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T07:50:27.133 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:26 smithi131 ceph-mon[29507]: from='mgr.? 172.21.15.157:0/1324886962' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T07:50:27.133 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:26 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:26.895+0000 7f36ee208200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T07:50:27.133 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:26.999+0000 7f36ee208200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T07:50:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:26 smithi157 ceph-mon[33345]: Standby manager daemon x restarted 2024-02-05T07:50:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:26 smithi157 ceph-mon[33345]: Standby manager daemon x started 2024-02-05T07:50:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:26 smithi157 ceph-mon[33345]: from='mgr.? 172.21.15.157:0/1324886962' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T07:50:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:26 smithi157 ceph-mon[33345]: from='mgr.? 172.21.15.157:0/1324886962' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T07:50:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:26 smithi157 ceph-mon[33345]: from='mgr.? 172.21.15.157:0/1324886962' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T07:50:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:26 smithi157 ceph-mon[33345]: from='mgr.? 172.21.15.157:0/1324886962' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T07:50:27.222 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:26 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:26.780+0000 7f1f9bba1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T07:50:27.222 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:26 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: [05/Feb/2024:07:50:26] ENGINE Bus STARTING 2024-02-05T07:50:27.222 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:26 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: CherryPy Checker: 2024-02-05T07:50:27.222 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:26 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: The Application mounted at '' has an empty config. 2024-02-05T07:50:27.222 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:26 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: 2024-02-05T07:50:27.222 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:26 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: [05/Feb/2024:07:50:26] ENGINE Serving on http://:::9283 2024-02-05T07:50:27.222 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 07:50:26 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x[35433]: [05/Feb/2024:07:50:26] ENGINE Bus STARTED 2024-02-05T07:50:27.385 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:27.233+0000 7f36ee208200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T07:50:27.385 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:50:27] ENGINE Bus STARTING 2024-02-05T07:50:27.720 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: CherryPy Checker: 2024-02-05T07:50:27.720 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: The Application mounted at '' has an empty config. 2024-02-05T07:50:27.720 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T07:50:27.720 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:50:27] ENGINE Serving on http://:::9283 2024-02-05T07:50:27.721 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:50:27] ENGINE Bus STARTED 2024-02-05T07:50:27.990 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: mgrmap e18: y(active, since 5m), standbys: x 2024-02-05T07:50:27.990 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: Active manager daemon y restarted 2024-02-05T07:50:27.990 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: Activating manager daemon y 2024-02-05T07:50:27.990 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: osdmap e74: 8 total, 8 up, 8 in 2024-02-05T07:50:27.990 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: mgrmap e19: y(active, starting, since 0.0101773s), standbys: x 2024-02-05T07:50:27.990 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:50:27.990 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:50:27.991 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:50:27.991 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T07:50:27.991 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T07:50:27.991 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:50:27.991 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:50:27.991 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:50:27.991 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:50:27.991 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:50:27.991 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:50:27.991 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:50:27.991 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:50:27.991 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T07:50:27.991 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T07:50:27.991 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T07:50:27.991 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: Manager daemon y is now available 2024-02-05T07:50:27.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:50:27.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:50:27.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:27.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:27.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:50:27.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:50:27.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:50:27.992 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:50:27.992 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: mgrmap e18: y(active, since 5m), standbys: x 2024-02-05T07:50:27.992 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: Active manager daemon y restarted 2024-02-05T07:50:27.992 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: Activating manager daemon y 2024-02-05T07:50:27.992 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: osdmap e74: 8 total, 8 up, 8 in 2024-02-05T07:50:27.993 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: mgrmap e19: y(active, starting, since 0.0101773s), standbys: x 2024-02-05T07:50:27.993 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:50:27.993 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:50:27.993 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:50:27.993 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T07:50:27.993 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T07:50:27.993 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:50:27.993 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:50:27.993 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:50:27.993 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:50:27.993 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:50:27.993 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:50:27.993 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:50:27.993 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:50:27.993 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T07:50:27.994 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T07:50:27.994 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T07:50:27.994 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: Manager daemon y is now available 2024-02-05T07:50:27.994 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:50:27.994 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:50:27.994 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:27.994 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:27.994 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:50:27.994 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:50:27.994 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:50:27.994 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:27 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:50:28.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: mgrmap e18: y(active, since 5m), standbys: x 2024-02-05T07:50:28.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: Active manager daemon y restarted 2024-02-05T07:50:28.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: Activating manager daemon y 2024-02-05T07:50:28.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: osdmap e74: 8 total, 8 up, 8 in 2024-02-05T07:50:28.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: mgrmap e19: y(active, starting, since 0.0101773s), standbys: x 2024-02-05T07:50:28.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: Manager daemon y is now available 2024-02-05T07:50:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:50:28.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:50:28.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:28.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:28.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:50:28.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:50:28.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:50:28.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:27 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:50:28.570 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:50:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:50:29.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:28 smithi131 ceph-mon[29507]: [05/Feb/2024:07:50:27] ENGINE Bus STARTING 2024-02-05T07:50:29.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:28 smithi131 ceph-mon[29507]: [05/Feb/2024:07:50:27] ENGINE Serving on http://172.21.15.131:8765 2024-02-05T07:50:29.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:28 smithi131 ceph-mon[29507]: [05/Feb/2024:07:50:27] ENGINE Serving on https://172.21.15.131:7150 2024-02-05T07:50:29.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:28 smithi131 ceph-mon[29507]: [05/Feb/2024:07:50:27] ENGINE Bus STARTED 2024-02-05T07:50:29.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:28 smithi131 ceph-mon[29507]: mgrmap e20: y(active, since 1.0142s), standbys: x 2024-02-05T07:50:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:28 smithi131 ceph-mon[41654]: [05/Feb/2024:07:50:27] ENGINE Bus STARTING 2024-02-05T07:50:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:28 smithi131 ceph-mon[41654]: [05/Feb/2024:07:50:27] ENGINE Serving on http://172.21.15.131:8765 2024-02-05T07:50:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:28 smithi131 ceph-mon[41654]: [05/Feb/2024:07:50:27] ENGINE Serving on https://172.21.15.131:7150 2024-02-05T07:50:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:28 smithi131 ceph-mon[41654]: [05/Feb/2024:07:50:27] ENGINE Bus STARTED 2024-02-05T07:50:29.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:28 smithi131 ceph-mon[41654]: mgrmap e20: y(active, since 1.0142s), standbys: x 2024-02-05T07:50:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:28 smithi157 ceph-mon[33345]: [05/Feb/2024:07:50:27] ENGINE Bus STARTING 2024-02-05T07:50:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:28 smithi157 ceph-mon[33345]: [05/Feb/2024:07:50:27] ENGINE Serving on http://172.21.15.131:8765 2024-02-05T07:50:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:28 smithi157 ceph-mon[33345]: [05/Feb/2024:07:50:27] ENGINE Serving on https://172.21.15.131:7150 2024-02-05T07:50:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:28 smithi157 ceph-mon[33345]: [05/Feb/2024:07:50:27] ENGINE Bus STARTED 2024-02-05T07:50:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:28 smithi157 ceph-mon[33345]: mgrmap e20: y(active, since 1.0142s), standbys: x 2024-02-05T07:50:29.929 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:29 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:29.929 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:29 smithi131 ceph-mon[41654]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:50:29.929 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:29 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:29.929 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:29 smithi131 ceph-mon[29507]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:50:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:29 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:29 smithi157 ceph-mon[33345]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:50:30.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:30 smithi157 ceph-mon[33345]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:50:30.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:30 smithi157 ceph-mon[33345]: mgrmap e21: y(active, since 2s), standbys: x 2024-02-05T07:50:31.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:30 smithi131 ceph-mon[41654]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:50:31.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:30 smithi131 ceph-mon[41654]: mgrmap e21: y(active, since 2s), standbys: x 2024-02-05T07:50:31.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:30 smithi131 ceph-mon[29507]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:50:31.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:30 smithi131 ceph-mon[29507]: mgrmap e21: y(active, since 2s), standbys: x 2024-02-05T07:50:31.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.471+0000 7f67f3dc4700 1 -- 172.21.15.131:0/2379238883 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f67ec102030 msgr2=0x7f67ec1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:31.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.471+0000 7f67f3dc4700 1 --2- 172.21.15.131:0/2379238883 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f67ec102030 0x7f67ec1058c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009990 tx=0x7f67e8009ca0 comp rx=0 tx=0).stop 2024-02-05T07:50:31.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.471+0000 7f67f3dc4700 1 -- 172.21.15.131:0/2379238883 shutdown_connections 2024-02-05T07:50:31.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.471+0000 7f67f3dc4700 1 --2- 172.21.15.131:0/2379238883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec101680 0x7f67ec101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:31.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.471+0000 7f67f3dc4700 1 --2- 172.21.15.131:0/2379238883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67ec105fa0 0x7f67ec10db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:31.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.471+0000 7f67f3dc4700 1 --2- 172.21.15.131:0/2379238883 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f67ec102030 0x7f67ec1058c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:31.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.471+0000 7f67f3dc4700 1 -- 172.21.15.131:0/2379238883 >> 172.21.15.131:0/2379238883 conn(0x7f67ec077220 msgr2=0x7f67ec0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:31.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.471+0000 7f67f3dc4700 1 -- 172.21.15.131:0/2379238883 shutdown_connections 2024-02-05T07:50:31.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.471+0000 7f67f3dc4700 1 -- 172.21.15.131:0/2379238883 wait complete. 2024-02-05T07:50:31.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.471+0000 7f67f3dc4700 1 Processor -- start 2024-02-05T07:50:31.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.471+0000 7f67f3dc4700 1 -- start start 2024-02-05T07:50:31.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f3dc4700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f67ec101680 0x7f67ec1965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:31.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f3dc4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec102030 0x7f67ec196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:31.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f3dc4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67ec105fa0 0x7f67ec19b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:31.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f3dc4700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec197420 con 0x7f67ec102030 2024-02-05T07:50:31.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f3dc4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec19bb70 con 0x7f67ec105fa0 2024-02-05T07:50:31.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f3dc4700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec19bd30 con 0x7f67ec101680 2024-02-05T07:50:31.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f1b60700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f67ec101680 0x7f67ec1965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:31.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f135f700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec102030 0x7f67ec196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:31.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f1b60700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f67ec101680 0x7f67ec1965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:42824/0 (socket says 172.21.15.131:42824) 2024-02-05T07:50:31.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f1b60700 1 -- 172.21.15.131:0/932192869 learned_addr learned my addr 172.21.15.131:0/932192869 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:31.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f2361700 1 --2- 172.21.15.131:0/932192869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67ec105fa0 0x7f67ec19b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:31.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f1b60700 1 -- 172.21.15.131:0/932192869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67ec105fa0 msgr2=0x7f67ec19b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:31.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f1b60700 1 --2- 172.21.15.131:0/932192869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67ec105fa0 0x7f67ec19b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:31.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f1b60700 1 -- 172.21.15.131:0/932192869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec102030 msgr2=0x7f67ec196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:31.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f1b60700 1 --2- 172.21.15.131:0/932192869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec102030 0x7f67ec196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:31.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f1b60700 1 -- 172.21.15.131:0/932192869 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc009520 con 0x7f67ec101680 2024-02-05T07:50:31.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f135f700 1 --2- 172.21.15.131:0/932192869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec102030 0x7f67ec196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:50:31.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f1b60700 1 --2- 172.21.15.131:0/932192869 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f67ec101680 0x7f67ec1965e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00eb30 tx=0x7f67dc00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:31.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67e2ffd700 1 -- 172.21.15.131:0/932192869 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc009880 con 0x7f67ec101680 2024-02-05T07:50:31.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67f3dc4700 1 -- 172.21.15.131:0/932192869 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e80095f0 con 0x7f67ec101680 2024-02-05T07:50:31.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67e2ffd700 1 -- 172.21.15.131:0/932192869 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f67dc0044d0 con 0x7f67ec101680 2024-02-05T07:50:31.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.472+0000 7f67e2ffd700 1 -- 172.21.15.131:0/932192869 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc003d10 con 0x7f67ec101680 2024-02-05T07:50:31.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.473+0000 7f67f3dc4700 1 -- 172.21.15.131:0/932192869 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ec19c340 con 0x7f67ec101680 2024-02-05T07:50:31.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.473+0000 7f67f3dc4700 1 -- 172.21.15.131:0/932192869 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f67ec1106d0 con 0x7f67ec101680 2024-02-05T07:50:31.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.474+0000 7f67e2ffd700 1 -- 172.21.15.131:0/932192869 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 21) v1 ==== 91374+0+0 (secure 0 0 0) 0x7f67dc003f70 con 0x7f67ec101680 2024-02-05T07:50:31.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.474+0000 7f67e2ffd700 1 --2- 172.21.15.131:0/932192869 >> v2:172.21.15.131:6800/2614238408 conn(0x7f67d806d9f0 0x7f67d806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:31.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.474+0000 7f67e2ffd700 1 -- 172.21.15.131:0/932192869 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f67dc014070 con 0x7f67ec101680 2024-02-05T07:50:31.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.474+0000 7f67f135f700 1 --2- 172.21.15.131:0/932192869 >> v2:172.21.15.131:6800/2614238408 conn(0x7f67d806d9f0 0x7f67d806feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:31.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.474+0000 7f67e2ffd700 1 --2- 172.21.15.131:0/932192869 >> v2:172.21.15.157:6812/1810544184 conn(0x7f67d8077310 0x7f67d80797b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:31.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.474+0000 7f67e2ffd700 1 -- 172.21.15.131:0/932192869 --> v2:172.21.15.157:6812/1810544184 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67d8079e40 con 0x7f67d8077310 2024-02-05T07:50:31.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.474+0000 7f67e2ffd700 1 -- 172.21.15.131:0/932192869 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67dc08fb50 con 0x7f67ec101680 2024-02-05T07:50:31.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.475+0000 7f67f2361700 1 --2- 172.21.15.131:0/932192869 >> v2:172.21.15.157:6812/1810544184 conn(0x7f67d8077310 0x7f67d80797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:31.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.475+0000 7f67f135f700 1 --2- 172.21.15.131:0/932192869 >> v2:172.21.15.131:6800/2614238408 conn(0x7f67d806d9f0 0x7f67d806feb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f67e800b2e0 tx=0x7f67e8005960 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:31.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.475+0000 7f67f2361700 1 --2- 172.21.15.131:0/932192869 >> v2:172.21.15.157:6812/1810544184 conn(0x7f67d8077310 0x7f67d80797b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:31.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.476+0000 7f67e2ffd700 1 -- 172.21.15.131:0/932192869 <== osd.7 v2:172.21.15.157:6812/1810544184 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f67d8079e40 con 0x7f67d8077310 2024-02-05T07:50:31.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.492+0000 7f67f3dc4700 1 -- 172.21.15.131:0/932192869 --> v2:172.21.15.157:6812/1810544184 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f67ec19c7e0 con 0x7f67d8077310 2024-02-05T07:50:31.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.492+0000 7f67e2ffd700 1 -- 172.21.15.131:0/932192869 <== osd.7 v2:172.21.15.157:6812/1810544184 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f67ec19c7e0 con 0x7f67d8077310 2024-02-05T07:50:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.492+0000 7f67f3dc4700 1 -- 172.21.15.131:0/932192869 >> v2:172.21.15.157:6812/1810544184 conn(0x7f67d8077310 msgr2=0x7f67d80797b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.492+0000 7f67f3dc4700 1 --2- 172.21.15.131:0/932192869 >> v2:172.21.15.157:6812/1810544184 conn(0x7f67d8077310 0x7f67d80797b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.492+0000 7f67f3dc4700 1 -- 172.21.15.131:0/932192869 >> v2:172.21.15.131:6800/2614238408 conn(0x7f67d806d9f0 msgr2=0x7f67d806feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.492+0000 7f67f3dc4700 1 --2- 172.21.15.131:0/932192869 >> v2:172.21.15.131:6800/2614238408 conn(0x7f67d806d9f0 0x7f67d806feb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f67e800b2e0 tx=0x7f67e8005960 comp rx=0 tx=0).stop 2024-02-05T07:50:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.492+0000 7f67f3dc4700 1 -- 172.21.15.131:0/932192869 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f67ec101680 msgr2=0x7f67ec1965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.492+0000 7f67f3dc4700 1 --2- 172.21.15.131:0/932192869 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f67ec101680 0x7f67ec1965e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00eb30 tx=0x7f67dc00ee40 comp rx=0 tx=0).stop 2024-02-05T07:50:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.493+0000 7f67f2361700 1 -- 172.21.15.131:0/932192869 reap_dead start 2024-02-05T07:50:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.493+0000 7f67f3dc4700 1 -- 172.21.15.131:0/932192869 shutdown_connections 2024-02-05T07:50:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.493+0000 7f67f3dc4700 1 -- 172.21.15.131:0/932192869 >> 172.21.15.131:0/932192869 conn(0x7f67ec077220 msgr2=0x7f67ec103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:31.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.493+0000 7f67f3dc4700 1 -- 172.21.15.131:0/932192869 shutdown_connections 2024-02-05T07:50:31.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.493+0000 7f67f3dc4700 1 -- 172.21.15.131:0/932192869 wait complete. 2024-02-05T07:50:31.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.659+0000 7f9302a6e700 1 -- 172.21.15.131:0/3979396870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92fc100eb0 msgr2=0x7f92fc101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:31.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.659+0000 7f9302a6e700 1 --2- 172.21.15.131:0/3979396870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92fc100eb0 0x7f92fc101290 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009990 tx=0x7f92f0009ca0 comp rx=0 tx=0).stop 2024-02-05T07:50:31.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.659+0000 7f9302a6e700 1 -- 172.21.15.131:0/3979396870 shutdown_connections 2024-02-05T07:50:31.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.659+0000 7f9302a6e700 1 --2- 172.21.15.131:0/3979396870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92fc100eb0 0x7f92fc101290 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.659+0000 7f9302a6e700 1 --2- 172.21.15.131:0/3979396870 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f92fc105f30 0x7f92fc10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.659+0000 7f9302a6e700 1 --2- 172.21.15.131:0/3979396870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92fc101860 0x7f92fc105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.659+0000 7f9302a6e700 1 -- 172.21.15.131:0/3979396870 >> 172.21.15.131:0/3979396870 conn(0x7f92fc076c30 msgr2=0x7f92fc077040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.659+0000 7f9302a6e700 1 -- 172.21.15.131:0/3979396870 shutdown_connections 2024-02-05T07:50:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.659+0000 7f9302a6e700 1 -- 172.21.15.131:0/3979396870 wait complete. 2024-02-05T07:50:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.660+0000 7f9302a6e700 1 Processor -- start 2024-02-05T07:50:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.660+0000 7f9302a6e700 1 -- start start 2024-02-05T07:50:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.660+0000 7f9302a6e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92fc100eb0 0x7f92fc19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.660+0000 7f9302a6e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92fc101860 0x7f92fc19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.660+0000 7f9302a6e700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f92fc105f30 0x7f92fc197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.660+0000 7f9302a6e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc197fc0 con 0x7f92fc101860 2024-02-05T07:50:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.660+0000 7f9302a6e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc198130 con 0x7f92fc100eb0 2024-02-05T07:50:31.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.660+0000 7f9302a6e700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc1982f0 con 0x7f92fc105f30 2024-02-05T07:50:31.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.660+0000 7f92fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92fc101860 0x7f92fc19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:31.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.660+0000 7f930100b700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f92fc105f30 0x7f92fc197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:31.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.660+0000 7f92fbfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92fc101860 0x7f92fc19dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33910/0 (socket says 172.21.15.131:33910) 2024-02-05T07:50:31.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.660+0000 7f92fbfff700 1 -- 172.21.15.131:0/3884839801 learned_addr learned my addr 172.21.15.131:0/3884839801 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:31.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.660+0000 7f930080a700 1 --2- 172.21.15.131:0/3884839801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92fc100eb0 0x7f92fc19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:31.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.661+0000 7f930100b700 1 -- 172.21.15.131:0/3884839801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92fc100eb0 msgr2=0x7f92fc19d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:31.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.661+0000 7f930100b700 1 --2- 172.21.15.131:0/3884839801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92fc100eb0 0x7f92fc19d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:31.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.661+0000 7f930100b700 1 -- 172.21.15.131:0/3884839801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92fc101860 msgr2=0x7f92fc19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:31.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.661+0000 7f930100b700 1 --2- 172.21.15.131:0/3884839801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92fc101860 0x7f92fc19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:31.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.661+0000 7f930100b700 1 -- 172.21.15.131:0/3884839801 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f00095f0 con 0x7f92fc105f30 2024-02-05T07:50:31.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.661+0000 7f92fbfff700 1 --2- 172.21.15.131:0/3884839801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92fc101860 0x7f92fc19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:50:31.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.661+0000 7f930100b700 1 --2- 172.21.15.131:0/3884839801 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f92fc105f30 0x7f92fc197810 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f92f400efc0 tx=0x7f92f400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:31.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.661+0000 7f92f9ffb700 1 -- 172.21.15.131:0/3884839801 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4018070 con 0x7f92fc105f30 2024-02-05T07:50:31.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.661+0000 7f92f9ffb700 1 -- 172.21.15.131:0/3884839801 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f92f400f040 con 0x7f92fc105f30 2024-02-05T07:50:31.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.661+0000 7f9302a6e700 1 -- 172.21.15.131:0/3884839801 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92fc198570 con 0x7f92fc105f30 2024-02-05T07:50:31.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.661+0000 7f92f9ffb700 1 -- 172.21.15.131:0/3884839801 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f400cc10 con 0x7f92fc105f30 2024-02-05T07:50:31.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.661+0000 7f9302a6e700 1 -- 172.21.15.131:0/3884839801 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92fc1a4620 con 0x7f92fc105f30 2024-02-05T07:50:31.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.662+0000 7f9302a6e700 1 -- 172.21.15.131:0/3884839801 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f92fc110650 con 0x7f92fc105f30 2024-02-05T07:50:31.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.662+0000 7f92f9ffb700 1 -- 172.21.15.131:0/3884839801 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 21) v1 ==== 91374+0+0 (secure 0 0 0) 0x7f92f4007500 con 0x7f92fc105f30 2024-02-05T07:50:31.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.663+0000 7f92f9ffb700 1 --2- 172.21.15.131:0/3884839801 >> v2:172.21.15.131:6800/2614238408 conn(0x7f92ec06d9f0 0x7f92ec06feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.663+0000 7f92f9ffb700 1 -- 172.21.15.131:0/3884839801 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f92f408cd80 con 0x7f92fc105f30 2024-02-05T07:50:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.663+0000 7f930080a700 1 --2- 172.21.15.131:0/3884839801 >> v2:172.21.15.131:6800/2614238408 conn(0x7f92ec06d9f0 0x7f92ec06feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.663+0000 7f92f9ffb700 1 --2- 172.21.15.131:0/3884839801 >> v2:172.21.15.157:6800/1828659015 conn(0x7f92ec077290 0x7f92ec079730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.663+0000 7f92f9ffb700 1 -- 172.21.15.131:0/3884839801 --> v2:172.21.15.157:6800/1828659015 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f92ec079dc0 con 0x7f92ec077290 2024-02-05T07:50:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.663+0000 7f92f9ffb700 1 -- 172.21.15.131:0/3884839801 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92f408f3e0 con 0x7f92fc105f30 2024-02-05T07:50:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.663+0000 7f930080a700 1 --2- 172.21.15.131:0/3884839801 >> v2:172.21.15.131:6800/2614238408 conn(0x7f92ec06d9f0 0x7f92ec06feb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f92f000b2e0 tx=0x7f92f0005960 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.663+0000 7f92fbfff700 1 --2- 172.21.15.131:0/3884839801 >> v2:172.21.15.157:6800/1828659015 conn(0x7f92ec077290 0x7f92ec079730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.664+0000 7f92fbfff700 1 --2- 172.21.15.131:0/3884839801 >> v2:172.21.15.157:6800/1828659015 conn(0x7f92ec077290 0x7f92ec079730 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.665+0000 7f92f9ffb700 1 -- 172.21.15.131:0/3884839801 <== osd.4 v2:172.21.15.157:6800/1828659015 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f92ec079dc0 con 0x7f92ec077290 2024-02-05T07:50:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.681+0000 7f9302a6e700 1 -- 172.21.15.131:0/3884839801 --> v2:172.21.15.157:6800/1828659015 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f92fc105620 con 0x7f92ec077290 2024-02-05T07:50:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.681+0000 7f92f9ffb700 1 -- 172.21.15.131:0/3884839801 <== osd.4 v2:172.21.15.157:6800/1828659015 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f92fc105620 con 0x7f92ec077290 2024-02-05T07:50:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.682+0000 7f9302a6e700 1 -- 172.21.15.131:0/3884839801 >> v2:172.21.15.157:6800/1828659015 conn(0x7f92ec077290 msgr2=0x7f92ec079730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.682+0000 7f9302a6e700 1 --2- 172.21.15.131:0/3884839801 >> v2:172.21.15.157:6800/1828659015 conn(0x7f92ec077290 0x7f92ec079730 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.682+0000 7f9302a6e700 1 -- 172.21.15.131:0/3884839801 >> v2:172.21.15.131:6800/2614238408 conn(0x7f92ec06d9f0 msgr2=0x7f92ec06feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.682+0000 7f9302a6e700 1 --2- 172.21.15.131:0/3884839801 >> v2:172.21.15.131:6800/2614238408 conn(0x7f92ec06d9f0 0x7f92ec06feb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f92f000b2e0 tx=0x7f92f0005960 comp rx=0 tx=0).stop 2024-02-05T07:50:31.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.682+0000 7f9302a6e700 1 -- 172.21.15.131:0/3884839801 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f92fc105f30 msgr2=0x7f92fc197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:31.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.682+0000 7f9302a6e700 1 --2- 172.21.15.131:0/3884839801 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f92fc105f30 0x7f92fc197810 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f92f400efc0 tx=0x7f92f400c3a0 comp rx=0 tx=0).stop 2024-02-05T07:50:31.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.682+0000 7f930100b700 1 -- 172.21.15.131:0/3884839801 reap_dead start 2024-02-05T07:50:31.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.682+0000 7f9302a6e700 1 -- 172.21.15.131:0/3884839801 shutdown_connections 2024-02-05T07:50:31.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.682+0000 7f9302a6e700 1 -- 172.21.15.131:0/3884839801 >> 172.21.15.131:0/3884839801 conn(0x7f92fc076c30 msgr2=0x7f92fc104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:31.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.682+0000 7f9302a6e700 1 -- 172.21.15.131:0/3884839801 shutdown_connections 2024-02-05T07:50:31.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:31.682+0000 7f9302a6e700 1 -- 172.21.15.131:0/3884839801 wait complete. 2024-02-05T07:50:32.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:31 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:32.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:31 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:32.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:31 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:32.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:31 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:32.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:31 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:32.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:31 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:32.398 INFO:teuthology.orchestra.run.smithi131.stdout:133143986204 2024-02-05T07:50:32.399 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-05T07:50:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:32 smithi157 ceph-mon[33345]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:50:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:32 smithi157 ceph-mon[33345]: mgrmap e22: y(active, since 4s), standbys: x 2024-02-05T07:50:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:32 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:32 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:32 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:32 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:32 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:32 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:32 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:32 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:32 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:32 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:32 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:50:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:50:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[29507]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:50:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[29507]: mgrmap e22: y(active, since 4s), standbys: x 2024-02-05T07:50:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[41654]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:50:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[41654]: mgrmap e22: y(active, since 4s), standbys: x 2024-02-05T07:50:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:50:33.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:32 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:50:34.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:33 smithi157 ceph-mon[33345]: Adjusting osd_memory_target on smithi157 to 3760M 2024-02-05T07:50:34.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:33 smithi131 ceph-mon[29507]: Adjusting osd_memory_target on smithi157 to 3760M 2024-02-05T07:50:34.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:33 smithi131 ceph-mon[41654]: Adjusting osd_memory_target on smithi157 to 3760M 2024-02-05T07:50:34.301 INFO:teuthology.orchestra.run.smithi131.stdout:240518168589 2024-02-05T07:50:34.301 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-05T07:50:35.115 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:34 smithi131 ceph-mon[41654]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 23 op/s 2024-02-05T07:50:35.115 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:34 smithi131 ceph-mon[29507]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 23 op/s 2024-02-05T07:50:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:34 smithi157 ceph-mon[33345]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 23 op/s 2024-02-05T07:50:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:36 smithi157 ceph-mon[33345]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-05T07:50:37.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:36 smithi131 ceph-mon[29507]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-05T07:50:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:36 smithi131 ceph-mon[41654]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-05T07:50:37.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.953+0000 7f2c12f4e700 1 -- 172.21.15.131:0/1856208637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c0c105f40 msgr2=0x7f2c0c10db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:37.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.953+0000 7f2c12f4e700 1 --2- 172.21.15.131:0/1856208637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c0c105f40 0x7f2c0c10db10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400b0a0 tx=0x7f2c0400b3b0 comp rx=0 tx=0).stop 2024-02-05T07:50:37.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.953+0000 7f2c12f4e700 1 -- 172.21.15.131:0/1856208637 shutdown_connections 2024-02-05T07:50:37.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.953+0000 7f2c12f4e700 1 --2- 172.21.15.131:0/1856208637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c0c105f40 0x7f2c0c10db10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:37.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.953+0000 7f2c12f4e700 1 --2- 172.21.15.131:0/1856208637 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f2c0c100eb0 0x7f2c0c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:37.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.953+0000 7f2c12f4e700 1 --2- 172.21.15.131:0/1856208637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c0c101860 0x7f2c0c105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:37.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.953+0000 7f2c12f4e700 1 -- 172.21.15.131:0/1856208637 >> 172.21.15.131:0/1856208637 conn(0x7f2c0c076c40 msgr2=0x7f2c0c077050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.953+0000 7f2c12f4e700 1 -- 172.21.15.131:0/1856208637 shutdown_connections 2024-02-05T07:50:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.953+0000 7f2c12f4e700 1 -- 172.21.15.131:0/1856208637 wait complete. 2024-02-05T07:50:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.953+0000 7f2c12f4e700 1 Processor -- start 2024-02-05T07:50:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.953+0000 7f2c12f4e700 1 -- start start 2024-02-05T07:50:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c12f4e700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f2c0c100eb0 0x7f2c0c19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c12f4e700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c0c101860 0x7f2c0c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c12f4e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c0c105f40 0x7f2c0c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c12f4e700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c197eb0 con 0x7f2c0c101860 2024-02-05T07:50:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c12f4e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c198020 con 0x7f2c0c105f40 2024-02-05T07:50:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c12f4e700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c1981e0 con 0x7f2c0c100eb0 2024-02-05T07:50:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c0bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c0c101860 0x7f2c0c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c10cea700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f2c0c100eb0 0x7f2c0c19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c0bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c0c101860 0x7f2c0c19dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33928/0 (socket says 172.21.15.131:33928) 2024-02-05T07:50:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c0bfff700 1 -- 172.21.15.131:0/1859135393 learned_addr learned my addr 172.21.15.131:0/1859135393 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:37.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c10cea700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f2c0c100eb0 0x7f2c0c19d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:42856/0 (socket says 172.21.15.131:42856) 2024-02-05T07:50:37.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c0bfff700 1 -- 172.21.15.131:0/1859135393 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f2c0c100eb0 msgr2=0x7f2c0c19d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:37.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c0bfff700 1 --2- 172.21.15.131:0/1859135393 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f2c0c100eb0 0x7f2c0c19d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:37.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c0bfff700 1 -- 172.21.15.131:0/1859135393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c0c105f40 msgr2=0x7f2c0c197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:37.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c0bfff700 1 --2- 172.21.15.131:0/1859135393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c0c105f40 0x7f2c0c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:37.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c0bfff700 1 -- 172.21.15.131:0/1859135393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c04009d00 con 0x7f2c0c101860 2024-02-05T07:50:37.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c0bfff700 1 --2- 172.21.15.131:0/1859135393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c0c101860 0x7f2c0c19dbc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800c9e0 tx=0x7f2bf800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:37.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c09ffb700 1 -- 172.21.15.131:0/1859135393 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8007aa0 con 0x7f2c0c101860 2024-02-05T07:50:37.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c12f4e700 1 -- 172.21.15.131:0/1859135393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0c1984c0 con 0x7f2c0c101860 2024-02-05T07:50:37.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c09ffb700 1 -- 172.21.15.131:0/1859135393 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f2bf8007c00 con 0x7f2c0c101860 2024-02-05T07:50:37.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c09ffb700 1 -- 172.21.15.131:0/1859135393 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8018660 con 0x7f2c0c101860 2024-02-05T07:50:37.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.954+0000 7f2c12f4e700 1 -- 172.21.15.131:0/1859135393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0c1a46b0 con 0x7f2c0c101860 2024-02-05T07:50:37.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.955+0000 7f2c12f4e700 1 -- 172.21.15.131:0/1859135393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2c0c1106a0 con 0x7f2c0c101860 2024-02-05T07:50:37.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.956+0000 7f2c09ffb700 1 -- 172.21.15.131:0/1859135393 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f2bf800f450 con 0x7f2c0c101860 2024-02-05T07:50:37.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.957+0000 7f2c09ffb700 1 --2- 172.21.15.131:0/1859135393 >> v2:172.21.15.131:6800/2614238408 conn(0x7f2bfc06da80 0x7f2bfc06ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:37.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.957+0000 7f2c09ffb700 1 -- 172.21.15.131:0/1859135393 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f2bf808d350 con 0x7f2c0c101860 2024-02-05T07:50:37.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.957+0000 7f2c10cea700 1 --2- 172.21.15.131:0/1859135393 >> v2:172.21.15.131:6800/2614238408 conn(0x7f2bfc06da80 0x7f2bfc06ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:37.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.957+0000 7f2c09ffb700 1 --2- 172.21.15.131:0/1859135393 >> v2:172.21.15.131:6813/3290626285 conn(0x7f2bfc077320 0x7f2bfc0797c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:37.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.957+0000 7f2c09ffb700 1 -- 172.21.15.131:0/1859135393 --> v2:172.21.15.131:6813/3290626285 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2bfc079e50 con 0x7f2bfc077320 2024-02-05T07:50:37.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.957+0000 7f2c09ffb700 1 -- 172.21.15.131:0/1859135393 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bf808fa10 con 0x7f2c0c101860 2024-02-05T07:50:37.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.957+0000 7f2c114eb700 1 --2- 172.21.15.131:0/1859135393 >> v2:172.21.15.131:6813/3290626285 conn(0x7f2bfc077320 0x7f2bfc0797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:37.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.957+0000 7f2c10cea700 1 --2- 172.21.15.131:0/1859135393 >> v2:172.21.15.131:6800/2614238408 conn(0x7f2bfc06da80 0x7f2bfc06ff40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009c60 tx=0x7f2c00009260 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:37.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.958+0000 7f2c114eb700 1 --2- 172.21.15.131:0/1859135393 >> v2:172.21.15.131:6813/3290626285 conn(0x7f2bfc077320 0x7f2bfc0797c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:37.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.958+0000 7f2c09ffb700 1 -- 172.21.15.131:0/1859135393 <== osd.3 v2:172.21.15.131:6813/3290626285 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2bfc079e50 con 0x7f2bfc077320 2024-02-05T07:50:37.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.974+0000 7f2c12f4e700 1 -- 172.21.15.131:0/1859135393 --> v2:172.21.15.131:6813/3290626285 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2c0c0623b0 con 0x7f2bfc077320 2024-02-05T07:50:37.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.974+0000 7f2c09ffb700 1 -- 172.21.15.131:0/1859135393 <== osd.3 v2:172.21.15.131:6813/3290626285 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2c0c0623b0 con 0x7f2bfc077320 2024-02-05T07:50:37.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.975+0000 7f2c12f4e700 1 -- 172.21.15.131:0/1859135393 >> v2:172.21.15.131:6813/3290626285 conn(0x7f2bfc077320 msgr2=0x7f2bfc0797c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:37.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.975+0000 7f2c12f4e700 1 --2- 172.21.15.131:0/1859135393 >> v2:172.21.15.131:6813/3290626285 conn(0x7f2bfc077320 0x7f2bfc0797c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:37.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.975+0000 7f2c12f4e700 1 -- 172.21.15.131:0/1859135393 >> v2:172.21.15.131:6800/2614238408 conn(0x7f2bfc06da80 msgr2=0x7f2bfc06ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:37.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.975+0000 7f2c12f4e700 1 --2- 172.21.15.131:0/1859135393 >> v2:172.21.15.131:6800/2614238408 conn(0x7f2bfc06da80 0x7f2bfc06ff40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009c60 tx=0x7f2c00009260 comp rx=0 tx=0).stop 2024-02-05T07:50:37.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.975+0000 7f2c12f4e700 1 -- 172.21.15.131:0/1859135393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c0c101860 msgr2=0x7f2c0c19dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:37.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.975+0000 7f2c12f4e700 1 --2- 172.21.15.131:0/1859135393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2c0c101860 0x7f2c0c19dbc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800c9e0 tx=0x7f2bf800ccf0 comp rx=0 tx=0).stop 2024-02-05T07:50:37.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.975+0000 7f2c114eb700 1 -- 172.21.15.131:0/1859135393 reap_dead start 2024-02-05T07:50:37.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.975+0000 7f2c12f4e700 1 -- 172.21.15.131:0/1859135393 shutdown_connections 2024-02-05T07:50:37.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.975+0000 7f2c12f4e700 1 -- 172.21.15.131:0/1859135393 >> 172.21.15.131:0/1859135393 conn(0x7f2c0c076c40 msgr2=0x7f2c0c06a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:37.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.975+0000 7f2c12f4e700 1 -- 172.21.15.131:0/1859135393 shutdown_connections 2024-02-05T07:50:37.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:36.975+0000 7f2c12f4e700 1 -- 172.21.15.131:0/1859135393 wait complete. 2024-02-05T07:50:37.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.915+0000 7ff3b1fd2700 1 -- 172.21.15.131:0/208700020 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff3ac102030 msgr2=0x7ff3ac1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:37.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.915+0000 7ff3b1fd2700 1 --2- 172.21.15.131:0/208700020 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff3ac102030 0x7ff3ac1058c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff394009990 tx=0x7ff394009ca0 comp rx=0 tx=0).stop 2024-02-05T07:50:37.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.915+0000 7ff3b1fd2700 1 -- 172.21.15.131:0/208700020 shutdown_connections 2024-02-05T07:50:37.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.915+0000 7ff3b1fd2700 1 --2- 172.21.15.131:0/208700020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3ac101680 0x7ff3ac101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:37.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.915+0000 7ff3b1fd2700 1 --2- 172.21.15.131:0/208700020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3ac105fa0 0x7ff3ac10db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:37.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.915+0000 7ff3b1fd2700 1 --2- 172.21.15.131:0/208700020 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff3ac102030 0x7ff3ac1058c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:37.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.915+0000 7ff3b1fd2700 1 -- 172.21.15.131:0/208700020 >> 172.21.15.131:0/208700020 conn(0x7ff3ac077220 msgr2=0x7ff3ac0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:37.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.915+0000 7ff3b1fd2700 1 -- 172.21.15.131:0/208700020 shutdown_connections 2024-02-05T07:50:37.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.915+0000 7ff3b1fd2700 1 -- 172.21.15.131:0/208700020 wait complete. 2024-02-05T07:50:37.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.916+0000 7ff3b1fd2700 1 Processor -- start 2024-02-05T07:50:37.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.916+0000 7ff3b1fd2700 1 -- start start 2024-02-05T07:50:37.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.916+0000 7ff3b1fd2700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff3ac101680 0x7ff3ac1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:37.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.916+0000 7ff3b1fd2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3ac102030 0x7ff3ac198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:37.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.916+0000 7ff3b1fd2700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3ac105fa0 0x7ff3ac1a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:37.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.916+0000 7ff3b1fd2700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac199680 con 0x7ff3ac105fa0 2024-02-05T07:50:37.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.916+0000 7ff3b1fd2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac1a3e60 con 0x7ff3ac102030 2024-02-05T07:50:37.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.916+0000 7ff3b1fd2700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac1a3fa0 con 0x7ff3ac101680 2024-02-05T07:50:37.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.916+0000 7ff3ab7fe700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff3ac101680 0x7ff3ac1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:37.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.916+0000 7ff3ab7fe700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff3ac101680 0x7ff3ac1988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:42884/0 (socket says 172.21.15.131:42884) 2024-02-05T07:50:37.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.916+0000 7ff3ab7fe700 1 -- 172.21.15.131:0/736585247 learned_addr learned my addr 172.21.15.131:0/736585247 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:37.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.916+0000 7ff3abfff700 1 --2- 172.21.15.131:0/736585247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3ac105fa0 0x7ff3ac1a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:37.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.917+0000 7ff3aaffd700 1 --2- 172.21.15.131:0/736585247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3ac102030 0x7ff3ac198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:37.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.917+0000 7ff3ab7fe700 1 -- 172.21.15.131:0/736585247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3ac102030 msgr2=0x7ff3ac198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:37.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.917+0000 7ff3ab7fe700 1 --2- 172.21.15.131:0/736585247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3ac102030 0x7ff3ac198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:37.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.917+0000 7ff3ab7fe700 1 -- 172.21.15.131:0/736585247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3ac105fa0 msgr2=0x7ff3ac1a3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:37.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.917+0000 7ff3ab7fe700 1 --2- 172.21.15.131:0/736585247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3ac105fa0 0x7ff3ac1a3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:37.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.917+0000 7ff3ab7fe700 1 -- 172.21.15.131:0/736585247 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a4009c40 con 0x7ff3ac101680 2024-02-05T07:50:37.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.917+0000 7ff3abfff700 1 --2- 172.21.15.131:0/736585247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3ac105fa0 0x7ff3ac1a3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:50:37.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.917+0000 7ff3ab7fe700 1 --2- 172.21.15.131:0/736585247 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff3ac101680 0x7ff3ac1988a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400c370 tx=0x7ff3a400c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:37.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.917+0000 7ff3a8ff9700 1 -- 172.21.15.131:0/736585247 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4017070 con 0x7ff3ac101680 2024-02-05T07:50:37.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.917+0000 7ff3a8ff9700 1 -- 172.21.15.131:0/736585247 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7ff3a400f040 con 0x7ff3ac101680 2024-02-05T07:50:37.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.917+0000 7ff3b1fd2700 1 -- 172.21.15.131:0/736585247 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3940095f0 con 0x7ff3ac101680 2024-02-05T07:50:37.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.917+0000 7ff3a8ff9700 1 -- 172.21.15.131:0/736585247 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a400e050 con 0x7ff3ac101680 2024-02-05T07:50:37.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.917+0000 7ff3b1fd2700 1 -- 172.21.15.131:0/736585247 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac1a4560 con 0x7ff3ac101680 2024-02-05T07:50:37.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.918+0000 7ff3b1fd2700 1 -- 172.21.15.131:0/736585247 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff3ac110730 con 0x7ff3ac101680 2024-02-05T07:50:37.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.918+0000 7ff3a8ff9700 1 -- 172.21.15.131:0/736585247 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7ff3a4003a00 con 0x7ff3ac101680 2024-02-05T07:50:37.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.919+0000 7ff3a8ff9700 1 --2- 172.21.15.131:0/736585247 >> v2:172.21.15.131:6800/2614238408 conn(0x7ff39806da80 0x7ff39806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:37.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.919+0000 7ff3a8ff9700 1 -- 172.21.15.131:0/736585247 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7ff3a408ce20 con 0x7ff3ac101680 2024-02-05T07:50:37.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.919+0000 7ff3aaffd700 1 --2- 172.21.15.131:0/736585247 >> v2:172.21.15.131:6800/2614238408 conn(0x7ff39806da80 0x7ff39806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:37.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.919+0000 7ff3a8ff9700 1 --2- 172.21.15.131:0/736585247 >> v2:172.21.15.131:6801/1614402360 conn(0x7ff398077320 0x7ff3980797c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:37.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.919+0000 7ff3a8ff9700 1 -- 172.21.15.131:0/736585247 --> v2:172.21.15.131:6801/1614402360 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff398079e50 con 0x7ff398077320 2024-02-05T07:50:37.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.919+0000 7ff3a8ff9700 1 -- 172.21.15.131:0/736585247 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3a408f4e0 con 0x7ff3ac101680 2024-02-05T07:50:37.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.919+0000 7ff3abfff700 1 --2- 172.21.15.131:0/736585247 >> v2:172.21.15.131:6801/1614402360 conn(0x7ff398077320 0x7ff3980797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:37.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.919+0000 7ff3aaffd700 1 --2- 172.21.15.131:0/736585247 >> v2:172.21.15.131:6800/2614238408 conn(0x7ff39806da80 0x7ff39806ff40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff394006010 tx=0x7ff394005e80 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:37.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.919+0000 7ff3abfff700 1 --2- 172.21.15.131:0/736585247 >> v2:172.21.15.131:6801/1614402360 conn(0x7ff398077320 0x7ff3980797c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:37.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.920+0000 7ff3a8ff9700 1 -- 172.21.15.131:0/736585247 <== osd.0 v2:172.21.15.131:6801/1614402360 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff398079e50 con 0x7ff398077320 2024-02-05T07:50:37.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.937+0000 7ff3b1fd2700 1 -- 172.21.15.131:0/736585247 --> v2:172.21.15.131:6801/1614402360 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff3ac104340 con 0x7ff398077320 2024-02-05T07:50:37.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.937+0000 7ff3a8ff9700 1 -- 172.21.15.131:0/736585247 <== osd.0 v2:172.21.15.131:6801/1614402360 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff3ac104340 con 0x7ff398077320 2024-02-05T07:50:37.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.937+0000 7ff3b1fd2700 1 -- 172.21.15.131:0/736585247 >> v2:172.21.15.131:6801/1614402360 conn(0x7ff398077320 msgr2=0x7ff3980797c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:37.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.937+0000 7ff3b1fd2700 1 --2- 172.21.15.131:0/736585247 >> v2:172.21.15.131:6801/1614402360 conn(0x7ff398077320 0x7ff3980797c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:37.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.937+0000 7ff3b1fd2700 1 -- 172.21.15.131:0/736585247 >> v2:172.21.15.131:6800/2614238408 conn(0x7ff39806da80 msgr2=0x7ff39806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:37.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.937+0000 7ff3b1fd2700 1 --2- 172.21.15.131:0/736585247 >> v2:172.21.15.131:6800/2614238408 conn(0x7ff39806da80 0x7ff39806ff40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff394006010 tx=0x7ff394005e80 comp rx=0 tx=0).stop 2024-02-05T07:50:37.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.937+0000 7ff3b1fd2700 1 -- 172.21.15.131:0/736585247 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff3ac101680 msgr2=0x7ff3ac1988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:37.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.937+0000 7ff3b1fd2700 1 --2- 172.21.15.131:0/736585247 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff3ac101680 0x7ff3ac1988a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400c370 tx=0x7ff3a400c680 comp rx=0 tx=0).stop 2024-02-05T07:50:37.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.937+0000 7ff3abfff700 1 -- 172.21.15.131:0/736585247 reap_dead start 2024-02-05T07:50:37.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.937+0000 7ff3b1fd2700 1 -- 172.21.15.131:0/736585247 shutdown_connections 2024-02-05T07:50:37.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.937+0000 7ff3b1fd2700 1 -- 172.21.15.131:0/736585247 >> 172.21.15.131:0/736585247 conn(0x7ff3ac077220 msgr2=0x7ff3ac1039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:37.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.938+0000 7ff3b1fd2700 1 -- 172.21.15.131:0/736585247 shutdown_connections 2024-02-05T07:50:37.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:37.938+0000 7ff3b1fd2700 1 -- 172.21.15.131:0/736585247 wait complete. 2024-02-05T07:50:38.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.461+0000 7fcdb35f6700 1 -- 172.21.15.131:0/1876518469 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fcdac100310 msgr2=0x7fcdac100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:38.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.461+0000 7fcdb35f6700 1 --2- 172.21.15.131:0/1876518469 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fcdac100310 0x7fcdac100790 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009990 tx=0x7fcd9c009ca0 comp rx=0 tx=0).stop 2024-02-05T07:50:38.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.461+0000 7fcdb35f6700 1 -- 172.21.15.131:0/1876518469 shutdown_connections 2024-02-05T07:50:38.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.461+0000 7fcdb35f6700 1 --2- 172.21.15.131:0/1876518469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdac106d90 0x7fcdac107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:38.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.461+0000 7fcdb35f6700 1 --2- 172.21.15.131:0/1876518469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdac100cd0 0x7fcdac10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:38.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.461+0000 7fcdb35f6700 1 --2- 172.21.15.131:0/1876518469 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fcdac100310 0x7fcdac100790 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:38.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.461+0000 7fcdb35f6700 1 -- 172.21.15.131:0/1876518469 >> 172.21.15.131:0/1876518469 conn(0x7fcdac077220 msgr2=0x7fcdac0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:38.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.461+0000 7fcdb35f6700 1 -- 172.21.15.131:0/1876518469 shutdown_connections 2024-02-05T07:50:38.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.461+0000 7fcdb35f6700 1 -- 172.21.15.131:0/1876518469 wait complete. 2024-02-05T07:50:38.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb35f6700 1 Processor -- start 2024-02-05T07:50:38.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb35f6700 1 -- start start 2024-02-05T07:50:38.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb35f6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdac100310 0x7fcdac19fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb35f6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdac100cd0 0x7fcdac19ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb35f6700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fcdac106d90 0x7fcdac199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb35f6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac19a2d0 con 0x7fcdac100310 2024-02-05T07:50:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb35f6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac19a440 con 0x7fcdac100cd0 2024-02-05T07:50:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb35f6700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac19a600 con 0x7fcdac106d90 2024-02-05T07:50:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb1392700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdac100310 0x7fcdac19fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb1b93700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fcdac106d90 0x7fcdac199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb1392700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdac100310 0x7fcdac19fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40320/0 (socket says 172.21.15.131:40320) 2024-02-05T07:50:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb1392700 1 -- 172.21.15.131:0/85030990 learned_addr learned my addr 172.21.15.131:0/85030990 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb0b91700 1 --2- 172.21.15.131:0/85030990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdac100cd0 0x7fcdac19ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb1b93700 1 -- 172.21.15.131:0/85030990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdac100cd0 msgr2=0x7fcdac19ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb1b93700 1 --2- 172.21.15.131:0/85030990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdac100cd0 0x7fcdac19ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb1b93700 1 -- 172.21.15.131:0/85030990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdac100310 msgr2=0x7fcdac19fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:38.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb1b93700 1 --2- 172.21.15.131:0/85030990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdac100310 0x7fcdac19fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:38.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb1b93700 1 -- 172.21.15.131:0/85030990 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda4009c40 con 0x7fcdac106d90 2024-02-05T07:50:38.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb1392700 1 --2- 172.21.15.131:0/85030990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdac100310 0x7fcdac19fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:50:38.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.462+0000 7fcdb0b91700 1 --2- 172.21.15.131:0/85030990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdac100cd0 0x7fcdac19ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-05T07:50:38.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.463+0000 7fcdb1b93700 1 --2- 172.21.15.131:0/85030990 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fcdac106d90 0x7fcdac199b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcda800c940 tx=0x7fcda800cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:38.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.463+0000 7fcda27fc700 1 -- 172.21.15.131:0/85030990 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda8004230 con 0x7fcdac106d90 2024-02-05T07:50:38.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.463+0000 7fcdb35f6700 1 -- 172.21.15.131:0/85030990 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9c0095f0 con 0x7fcdac106d90 2024-02-05T07:50:38.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.463+0000 7fcda27fc700 1 -- 172.21.15.131:0/85030990 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fcda8004390 con 0x7fcdac106d90 2024-02-05T07:50:38.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.463+0000 7fcda27fc700 1 -- 172.21.15.131:0/85030990 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda8003a60 con 0x7fcdac106d90 2024-02-05T07:50:38.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.463+0000 7fcdb35f6700 1 -- 172.21.15.131:0/85030990 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdac1a69d0 con 0x7fcdac106d90 2024-02-05T07:50:38.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.464+0000 7fcdb35f6700 1 -- 172.21.15.131:0/85030990 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcdac1118a0 con 0x7fcdac106d90 2024-02-05T07:50:38.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.464+0000 7fcda27fc700 1 -- 172.21.15.131:0/85030990 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fcda8003ca0 con 0x7fcdac106d90 2024-02-05T07:50:38.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.465+0000 7fcda27fc700 1 --2- 172.21.15.131:0/85030990 >> v2:172.21.15.131:6800/2614238408 conn(0x7fcd9806da80 0x7fcd9806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:38.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.465+0000 7fcda27fc700 1 -- 172.21.15.131:0/85030990 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fcda808cee0 con 0x7fcdac106d90 2024-02-05T07:50:38.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.465+0000 7fcdb1392700 1 --2- 172.21.15.131:0/85030990 >> v2:172.21.15.131:6800/2614238408 conn(0x7fcd9806da80 0x7fcd9806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:38.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.465+0000 7fcda27fc700 1 --2- 172.21.15.131:0/85030990 >> v2:172.21.15.131:6809/1463881636 conn(0x7fcd98077320 0x7fcd980797c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:38.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.465+0000 7fcda27fc700 1 -- 172.21.15.131:0/85030990 --> v2:172.21.15.131:6809/1463881636 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcd98079e50 con 0x7fcd98077320 2024-02-05T07:50:38.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.465+0000 7fcda27fc700 1 -- 172.21.15.131:0/85030990 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcda808f5a0 con 0x7fcdac106d90 2024-02-05T07:50:38.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.465+0000 7fcdb0b91700 1 --2- 172.21.15.131:0/85030990 >> v2:172.21.15.131:6809/1463881636 conn(0x7fcd98077320 0x7fcd980797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:38.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.465+0000 7fcdb1392700 1 --2- 172.21.15.131:0/85030990 >> v2:172.21.15.131:6800/2614238408 conn(0x7fcd9806da80 0x7fcd9806ff40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcda400e700 tx=0x7fcda400b040 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:38.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.465+0000 7fcdb0b91700 1 --2- 172.21.15.131:0/85030990 >> v2:172.21.15.131:6809/1463881636 conn(0x7fcd98077320 0x7fcd980797c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:38.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.466+0000 7fcda27fc700 1 -- 172.21.15.131:0/85030990 <== osd.2 v2:172.21.15.131:6809/1463881636 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcd98079e50 con 0x7fcd98077320 2024-02-05T07:50:38.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.482+0000 7fcdb35f6700 1 -- 172.21.15.131:0/85030990 --> v2:172.21.15.131:6809/1463881636 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcdac10d950 con 0x7fcd98077320 2024-02-05T07:50:38.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.482+0000 7fcda27fc700 1 -- 172.21.15.131:0/85030990 <== osd.2 v2:172.21.15.131:6809/1463881636 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcdac10d950 con 0x7fcd98077320 2024-02-05T07:50:38.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.483+0000 7fcdb35f6700 1 -- 172.21.15.131:0/85030990 >> v2:172.21.15.131:6809/1463881636 conn(0x7fcd98077320 msgr2=0x7fcd980797c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:38.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.483+0000 7fcdb35f6700 1 --2- 172.21.15.131:0/85030990 >> v2:172.21.15.131:6809/1463881636 conn(0x7fcd98077320 0x7fcd980797c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:38.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.483+0000 7fcdb35f6700 1 -- 172.21.15.131:0/85030990 >> v2:172.21.15.131:6800/2614238408 conn(0x7fcd9806da80 msgr2=0x7fcd9806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:38.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.483+0000 7fcdb35f6700 1 --2- 172.21.15.131:0/85030990 >> v2:172.21.15.131:6800/2614238408 conn(0x7fcd9806da80 0x7fcd9806ff40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcda400e700 tx=0x7fcda400b040 comp rx=0 tx=0).stop 2024-02-05T07:50:38.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.483+0000 7fcdb35f6700 1 -- 172.21.15.131:0/85030990 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fcdac106d90 msgr2=0x7fcdac199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:38.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.483+0000 7fcdb35f6700 1 --2- 172.21.15.131:0/85030990 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fcdac106d90 0x7fcdac199b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcda800c940 tx=0x7fcda800cc50 comp rx=0 tx=0).stop 2024-02-05T07:50:38.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.483+0000 7fcdb1b93700 1 -- 172.21.15.131:0/85030990 reap_dead start 2024-02-05T07:50:38.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.483+0000 7fcdb35f6700 1 -- 172.21.15.131:0/85030990 shutdown_connections 2024-02-05T07:50:38.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.483+0000 7fcdb35f6700 1 -- 172.21.15.131:0/85030990 >> 172.21.15.131:0/85030990 conn(0x7fcdac077220 msgr2=0x7fcdac10cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:38.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.483+0000 7fcdb35f6700 1 -- 172.21.15.131:0/85030990 shutdown_connections 2024-02-05T07:50:38.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:38.483+0000 7fcdb35f6700 1 -- 172.21.15.131:0/85030990 wait complete. 2024-02-05T07:50:38.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:50:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:50:39.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:38 smithi131 ceph-mon[29507]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-05T07:50:39.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:38 smithi131 ceph-mon[41654]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-05T07:50:39.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.130+0000 7f0ce58de700 1 -- 172.21.15.131:0/835823360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ce0100cd0 msgr2=0x7f0ce010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.130+0000 7f0ce58de700 1 --2- 172.21.15.131:0/835823360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ce0100cd0 0x7f0ce010ec30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009f90 tx=0x7f0cd800b330 comp rx=0 tx=0).stop 2024-02-05T07:50:39.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.130+0000 7f0ce58de700 1 -- 172.21.15.131:0/835823360 shutdown_connections 2024-02-05T07:50:39.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.130+0000 7f0ce58de700 1 --2- 172.21.15.131:0/835823360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ce0100cd0 0x7f0ce010ec30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.130+0000 7f0ce58de700 1 --2- 172.21.15.131:0/835823360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ce0106d90 0x7f0ce0107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.130+0000 7f0ce58de700 1 --2- 172.21.15.131:0/835823360 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f0ce0100310 0x7f0ce0100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.130+0000 7f0ce58de700 1 -- 172.21.15.131:0/835823360 >> 172.21.15.131:0/835823360 conn(0x7f0ce0077220 msgr2=0x7f0ce00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:39.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.130+0000 7f0ce58de700 1 -- 172.21.15.131:0/835823360 shutdown_connections 2024-02-05T07:50:39.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.130+0000 7f0ce58de700 1 -- 172.21.15.131:0/835823360 wait complete. 2024-02-05T07:50:39.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0ce58de700 1 Processor -- start 2024-02-05T07:50:39.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0ce58de700 1 -- start start 2024-02-05T07:50:39.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0ce58de700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ce0100310 0x7f0ce019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:39.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0ce58de700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f0ce0100cd0 0x7f0ce019dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:39.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0ce58de700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ce0106d90 0x7f0ce0197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:39.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0ce58de700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0197fc0 con 0x7f0ce0100310 2024-02-05T07:50:39.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0ce58de700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0198100 con 0x7f0ce0106d90 2024-02-05T07:50:39.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0ce58de700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce01982c0 con 0x7f0ce0100cd0 2024-02-05T07:50:39.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0cdeffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ce0100310 0x7f0ce019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:39.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0cde7fc700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f0ce0100cd0 0x7f0ce019dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:39.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0cdeffd700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ce0100310 0x7f0ce019d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40360/0 (socket says 172.21.15.131:40360) 2024-02-05T07:50:39.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0cdeffd700 1 -- 172.21.15.131:0/3596781764 learned_addr learned my addr 172.21.15.131:0/3596781764 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:39.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0cde7fc700 1 -- 172.21.15.131:0/3596781764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ce0106d90 msgr2=0x7f0ce0197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0cdf7fe700 1 --2- 172.21.15.131:0/3596781764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ce0106d90 0x7f0ce0197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:39.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0cde7fc700 1 --2- 172.21.15.131:0/3596781764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ce0106d90 0x7f0ce0197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0cde7fc700 1 -- 172.21.15.131:0/3596781764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ce0100310 msgr2=0x7f0ce019d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0cde7fc700 1 --2- 172.21.15.131:0/3596781764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ce0100310 0x7f0ce019d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0cde7fc700 1 -- 172.21.15.131:0/3596781764 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40095f0 con 0x7f0ce0100cd0 2024-02-05T07:50:39.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.131+0000 7f0cdeffd700 1 --2- 172.21.15.131:0/3596781764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ce0100310 0x7f0ce019d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:50:39.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.132+0000 7f0cde7fc700 1 --2- 172.21.15.131:0/3596781764 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f0ce0100cd0 0x7f0ce019dc90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400eb90 tx=0x7f0cd400eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:39.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.132+0000 7f0ce48dc700 1 -- 172.21.15.131:0/3596781764 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400ccf0 con 0x7f0ce0100cd0 2024-02-05T07:50:39.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.132+0000 7f0ce48dc700 1 -- 172.21.15.131:0/3596781764 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f0cd400ce50 con 0x7f0ce0100cd0 2024-02-05T07:50:39.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.132+0000 7f0ce58de700 1 -- 172.21.15.131:0/3596781764 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd8009c40 con 0x7f0ce0100cd0 2024-02-05T07:50:39.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.132+0000 7f0ce48dc700 1 -- 172.21.15.131:0/3596781764 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4010430 con 0x7f0ce0100cd0 2024-02-05T07:50:39.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.132+0000 7f0ce58de700 1 -- 172.21.15.131:0/3596781764 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce01a4620 con 0x7f0ce0100cd0 2024-02-05T07:50:39.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.132+0000 7f0ce58de700 1 -- 172.21.15.131:0/3596781764 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ce0111930 con 0x7f0ce0100cd0 2024-02-05T07:50:39.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.133+0000 7f0ce48dc700 1 -- 172.21.15.131:0/3596781764 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f0cd4004020 con 0x7f0ce0100cd0 2024-02-05T07:50:39.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.133+0000 7f0ce48dc700 1 --2- 172.21.15.131:0/3596781764 >> v2:172.21.15.131:6800/2614238408 conn(0x7f0cc806da80 0x7f0cc806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:39.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.134+0000 7f0cdeffd700 1 --2- 172.21.15.131:0/3596781764 >> v2:172.21.15.131:6800/2614238408 conn(0x7f0cc806da80 0x7f0cc806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:39.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.134+0000 7f0ce48dc700 1 -- 172.21.15.131:0/3596781764 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f0cd4014070 con 0x7f0ce0100cd0 2024-02-05T07:50:39.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.134+0000 7f0ce48dc700 1 --2- 172.21.15.131:0/3596781764 >> v2:172.21.15.131:6805/1256644882 conn(0x7f0cc8077320 0x7f0cc80797c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:39.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.134+0000 7f0ce48dc700 1 -- 172.21.15.131:0/3596781764 --> v2:172.21.15.131:6805/1256644882 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cc8079e50 con 0x7f0cc8077320 2024-02-05T07:50:39.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.134+0000 7f0ce48dc700 1 -- 172.21.15.131:0/3596781764 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cd408fdc0 con 0x7f0ce0100cd0 2024-02-05T07:50:39.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.134+0000 7f0cdf7fe700 1 --2- 172.21.15.131:0/3596781764 >> v2:172.21.15.131:6805/1256644882 conn(0x7f0cc8077320 0x7f0cc80797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:39.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.134+0000 7f0cdeffd700 1 --2- 172.21.15.131:0/3596781764 >> v2:172.21.15.131:6800/2614238408 conn(0x7f0cc806da80 0x7f0cc806ff40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0106680 tx=0x7f0cd0005f90 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:39.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.134+0000 7f0cdf7fe700 1 --2- 172.21.15.131:0/3596781764 >> v2:172.21.15.131:6805/1256644882 conn(0x7f0cc8077320 0x7f0cc80797c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:39.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.135+0000 7f0ce48dc700 1 -- 172.21.15.131:0/3596781764 <== osd.1 v2:172.21.15.131:6805/1256644882 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0cc8079e50 con 0x7f0cc8077320 2024-02-05T07:50:39.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.151+0000 7f0ce58de700 1 -- 172.21.15.131:0/3596781764 --> v2:172.21.15.131:6805/1256644882 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ce00623b0 con 0x7f0cc8077320 2024-02-05T07:50:39.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.151+0000 7f0ce48dc700 1 -- 172.21.15.131:0/3596781764 <== osd.1 v2:172.21.15.131:6805/1256644882 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0ce00623b0 con 0x7f0cc8077320 2024-02-05T07:50:39.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.152+0000 7f0ce58de700 1 -- 172.21.15.131:0/3596781764 >> v2:172.21.15.131:6805/1256644882 conn(0x7f0cc8077320 msgr2=0x7f0cc80797c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.152+0000 7f0ce58de700 1 --2- 172.21.15.131:0/3596781764 >> v2:172.21.15.131:6805/1256644882 conn(0x7f0cc8077320 0x7f0cc80797c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.152+0000 7f0ce58de700 1 -- 172.21.15.131:0/3596781764 >> v2:172.21.15.131:6800/2614238408 conn(0x7f0cc806da80 msgr2=0x7f0cc806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.152+0000 7f0ce58de700 1 --2- 172.21.15.131:0/3596781764 >> v2:172.21.15.131:6800/2614238408 conn(0x7f0cc806da80 0x7f0cc806ff40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0106680 tx=0x7f0cd0005f90 comp rx=0 tx=0).stop 2024-02-05T07:50:39.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.152+0000 7f0ce58de700 1 -- 172.21.15.131:0/3596781764 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f0ce0100cd0 msgr2=0x7f0ce019dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.152+0000 7f0ce58de700 1 --2- 172.21.15.131:0/3596781764 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f0ce0100cd0 0x7f0ce019dc90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400eb90 tx=0x7f0cd400eea0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.152+0000 7f0cdf7fe700 1 -- 172.21.15.131:0/3596781764 reap_dead start 2024-02-05T07:50:39.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.152+0000 7f0ce58de700 1 -- 172.21.15.131:0/3596781764 shutdown_connections 2024-02-05T07:50:39.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.152+0000 7f0ce58de700 1 -- 172.21.15.131:0/3596781764 >> 172.21.15.131:0/3596781764 conn(0x7f0ce0077220 msgr2=0x7f0ce00fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:39.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.152+0000 7f0ce58de700 1 -- 172.21.15.131:0/3596781764 shutdown_connections 2024-02-05T07:50:39.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.152+0000 7f0ce58de700 1 -- 172.21.15.131:0/3596781764 wait complete. 2024-02-05T07:50:39.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:38 smithi157 ceph-mon[33345]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-05T07:50:39.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.059+0000 7f7eee437700 1 -- 172.21.15.131:0/658706168 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f7ee806ae70 msgr2=0x7f7ee8106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.059+0000 7f7eee437700 1 --2- 172.21.15.131:0/658706168 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f7ee806ae70 0x7f7ee8106970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80098f0 tx=0x7f7ed8009c00 comp rx=0 tx=0).stop 2024-02-05T07:50:39.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.059+0000 7f7eee437700 1 -- 172.21.15.131:0/658706168 shutdown_connections 2024-02-05T07:50:39.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.059+0000 7f7eee437700 1 --2- 172.21.15.131:0/658706168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ee81070a0 0x7f7ee810ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.059+0000 7f7eee437700 1 --2- 172.21.15.131:0/658706168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ee806a4c0 0x7f7ee806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.059+0000 7f7eee437700 1 --2- 172.21.15.131:0/658706168 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f7ee806ae70 0x7f7ee8106970 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.059+0000 7f7eee437700 1 -- 172.21.15.131:0/658706168 >> 172.21.15.131:0/658706168 conn(0x7f7ee8076c70 msgr2=0x7f7ee8077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:39.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.059+0000 7f7eee437700 1 -- 172.21.15.131:0/658706168 shutdown_connections 2024-02-05T07:50:39.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.059+0000 7f7eee437700 1 -- 172.21.15.131:0/658706168 wait complete. 2024-02-05T07:50:39.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.059+0000 7f7eee437700 1 Processor -- start 2024-02-05T07:50:39.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.059+0000 7f7eee437700 1 -- start start 2024-02-05T07:50:39.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7eee437700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ee806a4c0 0x7f7ee819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:39.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7eee437700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ee806ae70 0x7f7ee819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:39.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7eee437700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f7ee81070a0 0x7f7ee8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:39.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7eee437700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee819a090 con 0x7f7ee806a4c0 2024-02-05T07:50:39.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7eee437700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee819a1d0 con 0x7f7ee806ae70 2024-02-05T07:50:39.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7eee437700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee819a390 con 0x7f7ee81070a0 2024-02-05T07:50:39.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7ee7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ee806a4c0 0x7f7ee819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:39.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7eec9d4700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f7ee81070a0 0x7f7ee8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:39.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7ee7fff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ee806a4c0 0x7f7ee819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40350/0 (socket says 172.21.15.131:40350) 2024-02-05T07:50:39.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7ee7fff700 1 -- 172.21.15.131:0/1605120482 learned_addr learned my addr 172.21.15.131:0/1605120482 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:39.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7ee77fe700 1 --2- 172.21.15.131:0/1605120482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ee806ae70 0x7f7ee819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:39.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7eec9d4700 1 -- 172.21.15.131:0/1605120482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ee806ae70 msgr2=0x7f7ee819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7eec9d4700 1 --2- 172.21.15.131:0/1605120482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ee806ae70 0x7f7ee819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7eec9d4700 1 -- 172.21.15.131:0/1605120482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ee806a4c0 msgr2=0x7f7ee819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7eec9d4700 1 --2- 172.21.15.131:0/1605120482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ee806a4c0 0x7f7ee819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7eec9d4700 1 -- 172.21.15.131:0/1605120482 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee0009d00 con 0x7f7ee81070a0 2024-02-05T07:50:39.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7ee7fff700 1 --2- 172.21.15.131:0/1605120482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ee806a4c0 0x7f7ee819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:50:39.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7eec9d4700 1 --2- 172.21.15.131:0/1605120482 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f7ee81070a0 0x7f7ee8199a30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009cd0 tx=0x7f7ee000c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:39.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7ee57fa700 1 -- 172.21.15.131:0/1605120482 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0017070 con 0x7f7ee81070a0 2024-02-05T07:50:39.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7ee57fa700 1 -- 172.21.15.131:0/1605120482 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f7ee000f040 con 0x7f7ee81070a0 2024-02-05T07:50:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.060+0000 7f7ee57fa700 1 -- 172.21.15.131:0/1605120482 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee000e050 con 0x7f7ee81070a0 2024-02-05T07:50:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.061+0000 7f7eee437700 1 -- 172.21.15.131:0/1605120482 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed8009520 con 0x7f7ee81070a0 2024-02-05T07:50:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.061+0000 7f7eee437700 1 -- 172.21.15.131:0/1605120482 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee81a66f0 con 0x7f7ee81070a0 2024-02-05T07:50:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.062+0000 7f7eee437700 1 -- 172.21.15.131:0/1605120482 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7ee81117e0 con 0x7f7ee81070a0 2024-02-05T07:50:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.062+0000 7f7ee57fa700 1 -- 172.21.15.131:0/1605120482 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f7ee00090d0 con 0x7f7ee81070a0 2024-02-05T07:50:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.063+0000 7f7ee57fa700 1 --2- 172.21.15.131:0/1605120482 >> v2:172.21.15.131:6800/2614238408 conn(0x7f7ed006da80 0x7f7ed006ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.063+0000 7f7ee57fa700 1 -- 172.21.15.131:0/1605120482 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f7ee008cd40 con 0x7f7ee81070a0 2024-02-05T07:50:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.063+0000 7f7ee7fff700 1 --2- 172.21.15.131:0/1605120482 >> v2:172.21.15.131:6800/2614238408 conn(0x7f7ed006da80 0x7f7ed006ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.063+0000 7f7ee57fa700 1 --2- 172.21.15.131:0/1605120482 >> v2:172.21.15.157:6808/204226792 conn(0x7f7ed0077320 0x7f7ed00797c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.063+0000 7f7ee57fa700 1 -- 172.21.15.131:0/1605120482 --> v2:172.21.15.157:6808/204226792 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7ed0079e50 con 0x7f7ed0077320 2024-02-05T07:50:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.063+0000 7f7ee57fa700 1 -- 172.21.15.131:0/1605120482 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ee008f400 con 0x7f7ee81070a0 2024-02-05T07:50:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.063+0000 7f7ee7fff700 1 --2- 172.21.15.131:0/1605120482 >> v2:172.21.15.131:6800/2614238408 conn(0x7f7ed006da80 0x7f7ed006ff40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7edc005950 tx=0x7f7edc00a400 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:39.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.063+0000 7f7ee77fe700 1 --2- 172.21.15.131:0/1605120482 >> v2:172.21.15.157:6808/204226792 conn(0x7f7ed0077320 0x7f7ed00797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.064+0000 7f7ee77fe700 1 --2- 172.21.15.131:0/1605120482 >> v2:172.21.15.157:6808/204226792 conn(0x7f7ed0077320 0x7f7ed00797c0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.065+0000 7f7ee57fa700 1 -- 172.21.15.131:0/1605120482 <== osd.6 v2:172.21.15.157:6808/204226792 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7ed0079e50 con 0x7f7ed0077320 2024-02-05T07:50:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.081+0000 7f7eee437700 1 -- 172.21.15.131:0/1605120482 --> v2:172.21.15.157:6808/204226792 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7ee810d8a0 con 0x7f7ed0077320 2024-02-05T07:50:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.081+0000 7f7ee57fa700 1 -- 172.21.15.131:0/1605120482 <== osd.6 v2:172.21.15.157:6808/204226792 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7ee810d8a0 con 0x7f7ed0077320 2024-02-05T07:50:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.082+0000 7f7eee437700 1 -- 172.21.15.131:0/1605120482 >> v2:172.21.15.157:6808/204226792 conn(0x7f7ed0077320 msgr2=0x7f7ed00797c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.082+0000 7f7eee437700 1 --2- 172.21.15.131:0/1605120482 >> v2:172.21.15.157:6808/204226792 conn(0x7f7ed0077320 0x7f7ed00797c0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.082+0000 7f7eee437700 1 -- 172.21.15.131:0/1605120482 >> v2:172.21.15.131:6800/2614238408 conn(0x7f7ed006da80 msgr2=0x7f7ed006ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.082+0000 7f7eee437700 1 --2- 172.21.15.131:0/1605120482 >> v2:172.21.15.131:6800/2614238408 conn(0x7f7ed006da80 0x7f7ed006ff40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7edc005950 tx=0x7f7edc00a400 comp rx=0 tx=0).stop 2024-02-05T07:50:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.082+0000 7f7eee437700 1 -- 172.21.15.131:0/1605120482 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f7ee81070a0 msgr2=0x7f7ee8199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.082+0000 7f7eee437700 1 --2- 172.21.15.131:0/1605120482 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f7ee81070a0 0x7f7ee8199a30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009cd0 tx=0x7f7ee000c620 comp rx=0 tx=0).stop 2024-02-05T07:50:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.082+0000 7f7eec9d4700 1 -- 172.21.15.131:0/1605120482 reap_dead start 2024-02-05T07:50:39.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.082+0000 7f7eee437700 1 -- 172.21.15.131:0/1605120482 shutdown_connections 2024-02-05T07:50:39.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.082+0000 7f7eee437700 1 -- 172.21.15.131:0/1605120482 >> 172.21.15.131:0/1605120482 conn(0x7f7ee8076c70 msgr2=0x7f7ee810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:39.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.082+0000 7f7eee437700 1 -- 172.21.15.131:0/1605120482 shutdown_connections 2024-02-05T07:50:39.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.082+0000 7f7eee437700 1 -- 172.21.15.131:0/1605120482 wait complete. 2024-02-05T07:50:39.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.890+0000 7f9c24f64700 1 -- 172.21.15.131:0/1324358468 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9c2010e7d0 msgr2=0x7f9c20110bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.890+0000 7f9c24f64700 1 --2- 172.21.15.131:0/1324358468 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9c2010e7d0 0x7f9c20110bc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800b0a0 tx=0x7f9c1800b3b0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.890+0000 7f9c24f64700 1 -- 172.21.15.131:0/1324358468 shutdown_connections 2024-02-05T07:50:39.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.890+0000 7f9c24f64700 1 --2- 172.21.15.131:0/1324358468 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9c2010e7d0 0x7f9c20110bc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.890+0000 7f9c24f64700 1 --2- 172.21.15.131:0/1324358468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c200699d0 0x7f9c20069db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.890+0000 7f9c24f64700 1 --2- 172.21.15.131:0/1324358468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c2006a2f0 0x7f9c2010e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.890+0000 7f9c24f64700 1 -- 172.21.15.131:0/1324358468 >> 172.21.15.131:0/1324358468 conn(0x7f9c20077b50 msgr2=0x7f9c20077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:39.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.890+0000 7f9c24f64700 1 -- 172.21.15.131:0/1324358468 shutdown_connections 2024-02-05T07:50:39.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.890+0000 7f9c24f64700 1 -- 172.21.15.131:0/1324358468 wait complete. 2024-02-05T07:50:39.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c24f64700 1 Processor -- start 2024-02-05T07:50:39.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c24f64700 1 -- start start 2024-02-05T07:50:39.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c24f64700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c200699d0 0x7f9c2019fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:39.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c24f64700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9c2006a2f0 0x7f9c2019ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:39.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c24f64700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c2010e7d0 0x7f9c20199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:39.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c24f64700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2019a2d0 con 0x7f9c200699d0 2024-02-05T07:50:39.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c24f64700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2019a440 con 0x7f9c2010e7d0 2024-02-05T07:50:39.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c24f64700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2019a600 con 0x7f9c2006a2f0 2024-02-05T07:50:39.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c1dd9b700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9c2006a2f0 0x7f9c2019ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:39.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c1e59c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c200699d0 0x7f9c2019fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:39.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c1dd9b700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9c2006a2f0 0x7f9c2019ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:38826/0 (socket says 172.21.15.131:38826) 2024-02-05T07:50:39.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c1dd9b700 1 -- 172.21.15.131:0/2978599862 learned_addr learned my addr 172.21.15.131:0/2978599862 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:39.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c1ed9d700 1 --2- 172.21.15.131:0/2978599862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c2010e7d0 0x7f9c20199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:39.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c1e59c700 1 -- 172.21.15.131:0/2978599862 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9c2006a2f0 msgr2=0x7f9c2019ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c1e59c700 1 --2- 172.21.15.131:0/2978599862 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9c2006a2f0 0x7f9c2019ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c1e59c700 1 -- 172.21.15.131:0/2978599862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c2010e7d0 msgr2=0x7f9c20199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c1e59c700 1 --2- 172.21.15.131:0/2978599862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c2010e7d0 0x7f9c20199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c1e59c700 1 -- 172.21.15.131:0/2978599862 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c14009520 con 0x7f9c200699d0 2024-02-05T07:50:39.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c1dd9b700 1 --2- 172.21.15.131:0/2978599862 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f9c2006a2f0 0x7f9c2019ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:50:39.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.891+0000 7f9c1ed9d700 1 --2- 172.21.15.131:0/2978599862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c2010e7d0 0x7f9c20199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-05T07:50:39.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.892+0000 7f9c1e59c700 1 --2- 172.21.15.131:0/2978599862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c200699d0 0x7f9c2019fa50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000b450 tx=0x7f9c1000b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:39.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.892+0000 7f9c0f7fe700 1 -- 172.21.15.131:0/2978599862 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10011830 con 0x7f9c200699d0 2024-02-05T07:50:39.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.892+0000 7f9c24f64700 1 -- 172.21.15.131:0/2978599862 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c18009d00 con 0x7f9c200699d0 2024-02-05T07:50:39.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.892+0000 7f9c0f7fe700 1 -- 172.21.15.131:0/2978599862 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f9c10011e40 con 0x7f9c200699d0 2024-02-05T07:50:39.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.892+0000 7f9c0f7fe700 1 -- 172.21.15.131:0/2978599862 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10010de0 con 0x7f9c200699d0 2024-02-05T07:50:39.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.892+0000 7f9c24f64700 1 -- 172.21.15.131:0/2978599862 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c201a69d0 con 0x7f9c200699d0 2024-02-05T07:50:39.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.893+0000 7f9c24f64700 1 -- 172.21.15.131:0/2978599862 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9c20113960 con 0x7f9c200699d0 2024-02-05T07:50:39.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.893+0000 7f9c0f7fe700 1 -- 172.21.15.131:0/2978599862 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f9c10011990 con 0x7f9c200699d0 2024-02-05T07:50:39.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.894+0000 7f9c0f7fe700 1 --2- 172.21.15.131:0/2978599862 >> v2:172.21.15.131:6800/2614238408 conn(0x7f9c0806da80 0x7f9c0806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:39.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.894+0000 7f9c0f7fe700 1 -- 172.21.15.131:0/2978599862 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f9c1008ce70 con 0x7f9c200699d0 2024-02-05T07:50:39.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.894+0000 7f9c1dd9b700 1 --2- 172.21.15.131:0/2978599862 >> v2:172.21.15.131:6800/2614238408 conn(0x7f9c0806da80 0x7f9c0806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:39.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.894+0000 7f9c0f7fe700 1 --2- 172.21.15.131:0/2978599862 >> v2:172.21.15.157:6804/3589104149 conn(0x7f9c08077320 0x7f9c080797c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:39.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.894+0000 7f9c0f7fe700 1 -- 172.21.15.131:0/2978599862 --> v2:172.21.15.157:6804/3589104149 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c08079e50 con 0x7f9c08077320 2024-02-05T07:50:39.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.894+0000 7f9c0f7fe700 1 -- 172.21.15.131:0/2978599862 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c1008f530 con 0x7f9c200699d0 2024-02-05T07:50:39.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.894+0000 7f9c1dd9b700 1 --2- 172.21.15.131:0/2978599862 >> v2:172.21.15.131:6800/2614238408 conn(0x7f9c0806da80 0x7f9c0806ff40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9c14009fd0 tx=0x7f9c14009260 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:39.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.894+0000 7f9c1ed9d700 1 --2- 172.21.15.131:0/2978599862 >> v2:172.21.15.157:6804/3589104149 conn(0x7f9c08077320 0x7f9c080797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:39.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.895+0000 7f9c1ed9d700 1 --2- 172.21.15.131:0/2978599862 >> v2:172.21.15.157:6804/3589104149 conn(0x7f9c08077320 0x7f9c080797c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:39.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.896+0000 7f9c0f7fe700 1 -- 172.21.15.131:0/2978599862 <== osd.5 v2:172.21.15.157:6804/3589104149 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9c08079e50 con 0x7f9c08077320 2024-02-05T07:50:39.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.918+0000 7f9c24f64700 1 -- 172.21.15.131:0/2978599862 --> v2:172.21.15.157:6804/3589104149 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9c20100570 con 0x7f9c08077320 2024-02-05T07:50:39.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.919+0000 7f9c0f7fe700 1 -- 172.21.15.131:0/2978599862 <== osd.5 v2:172.21.15.157:6804/3589104149 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9c20100570 con 0x7f9c08077320 2024-02-05T07:50:39.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.920+0000 7f9c24f64700 1 -- 172.21.15.131:0/2978599862 >> v2:172.21.15.157:6804/3589104149 conn(0x7f9c08077320 msgr2=0x7f9c080797c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.920+0000 7f9c24f64700 1 --2- 172.21.15.131:0/2978599862 >> v2:172.21.15.157:6804/3589104149 conn(0x7f9c08077320 0x7f9c080797c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:39.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.920+0000 7f9c24f64700 1 -- 172.21.15.131:0/2978599862 >> v2:172.21.15.131:6800/2614238408 conn(0x7f9c0806da80 msgr2=0x7f9c0806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.920+0000 7f9c24f64700 1 --2- 172.21.15.131:0/2978599862 >> v2:172.21.15.131:6800/2614238408 conn(0x7f9c0806da80 0x7f9c0806ff40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9c14009fd0 tx=0x7f9c14009260 comp rx=0 tx=0).stop 2024-02-05T07:50:39.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.920+0000 7f9c24f64700 1 -- 172.21.15.131:0/2978599862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c200699d0 msgr2=0x7f9c2019fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:39.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.920+0000 7f9c24f64700 1 --2- 172.21.15.131:0/2978599862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c200699d0 0x7f9c2019fa50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000b450 tx=0x7f9c1000b760 comp rx=0 tx=0).stop 2024-02-05T07:50:39.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.920+0000 7f9c1ed9d700 1 -- 172.21.15.131:0/2978599862 reap_dead start 2024-02-05T07:50:39.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.920+0000 7f9c24f64700 1 -- 172.21.15.131:0/2978599862 shutdown_connections 2024-02-05T07:50:39.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.920+0000 7f9c24f64700 1 -- 172.21.15.131:0/2978599862 >> 172.21.15.131:0/2978599862 conn(0x7f9c20077b50 msgr2=0x7f9c200ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:39.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.920+0000 7f9c24f64700 1 -- 172.21.15.131:0/2978599862 shutdown_connections 2024-02-05T07:50:39.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:39.920+0000 7f9c24f64700 1 -- 172.21.15.131:0/2978599862 wait complete. 2024-02-05T07:50:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:39 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:39 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:40.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:39 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:40.805 INFO:teuthology.orchestra.run.smithi131.stdout:55834574894 2024-02-05T07:50:40.805 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-05T07:50:41.036 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:40 smithi131 ceph-mon[41654]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-05T07:50:41.036 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:40 smithi131 ceph-mon[29507]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-05T07:50:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:40 smithi157 ceph-mon[33345]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-05T07:50:41.355 INFO:teuthology.orchestra.run.smithi131.stdout:111669149731 2024-02-05T07:50:41.356 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-05T07:50:41.764 INFO:teuthology.orchestra.run.smithi131.stdout:201863462932 2024-02-05T07:50:41.764 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-05T07:50:42.198 INFO:teuthology.orchestra.run.smithi131.stdout:34359738418 2024-02-05T07:50:42.199 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-05T07:50:42.646 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:42.646 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:42.654 INFO:teuthology.orchestra.run.smithi131.stdout:167503724570 2024-02-05T07:50:42.654 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-05T07:50:43.106 INFO:teuthology.orchestra.run.smithi131.stdout:77309411369 2024-02-05T07:50:43.106 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-05T07:50:43.132 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:42 smithi131 ceph-mon[41654]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T07:50:43.132 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:42 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:43.132 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:42 smithi131 ceph-mon[29507]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T07:50:43.132 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:42 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:43.133 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:50:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:50:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:42 smithi157 ceph-mon[33345]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T07:50:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:42 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:43.817 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:44 smithi131 ceph-mon[41654]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T07:50:45.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:44 smithi131 ceph-mon[29507]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T07:50:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:44 smithi157 ceph-mon[33345]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T07:50:45.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.932+0000 7f1cd2759700 1 -- 172.21.15.131:0/3706415606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ccc074040 msgr2=0x7f1ccc074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:45.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.932+0000 7f1cd2759700 1 --2- 172.21.15.131:0/3706415606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ccc074040 0x7f1ccc074420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0098f0 tx=0x7f1cbc009c00 comp rx=0 tx=0).stop 2024-02-05T07:50:45.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.932+0000 7f1cd2759700 1 -- 172.21.15.131:0/3706415606 shutdown_connections 2024-02-05T07:50:45.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.932+0000 7f1cd2759700 1 --2- 172.21.15.131:0/3706415606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ccc074040 0x7f1ccc074420 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:45.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.932+0000 7f1cd2759700 1 --2- 172.21.15.131:0/3706415606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ccc10e5c0 0x7f1ccc1109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:45.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.932+0000 7f1cd2759700 1 --2- 172.21.15.131:0/3706415606 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1ccc074960 0x7f1ccc10e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:45.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.932+0000 7f1cd2759700 1 -- 172.21.15.131:0/3706415606 >> 172.21.15.131:0/3706415606 conn(0x7f1ccc0fd7d0 msgr2=0x7f1ccc0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:45.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.932+0000 7f1cd2759700 1 -- 172.21.15.131:0/3706415606 shutdown_connections 2024-02-05T07:50:45.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.932+0000 7f1cd2759700 1 -- 172.21.15.131:0/3706415606 wait complete. 2024-02-05T07:50:45.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.933+0000 7f1cd2759700 1 Processor -- start 2024-02-05T07:50:45.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.933+0000 7f1cd2759700 1 -- start start 2024-02-05T07:50:45.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.933+0000 7f1cd2759700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1ccc074040 0x7f1ccc19f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:45.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.933+0000 7f1cd2759700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ccc074960 0x7f1ccc19fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:45.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.933+0000 7f1cd2759700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ccc10e5c0 0x7f1ccc199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:45.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.933+0000 7f1cd2759700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc19a140 con 0x7f1ccc074960 2024-02-05T07:50:45.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.933+0000 7f1cd2759700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc19a280 con 0x7f1ccc10e5c0 2024-02-05T07:50:45.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.933+0000 7f1cd2759700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc19a440 con 0x7f1ccc074040 2024-02-05T07:50:45.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.933+0000 7f1ccb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ccc074960 0x7f1ccc19fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:45.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.933+0000 7f1ccbfff700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1ccc074040 0x7f1ccc19f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:45.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.933+0000 7f1ccb7fe700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ccc074960 0x7f1ccc19fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40412/0 (socket says 172.21.15.131:40412) 2024-02-05T07:50:45.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.933+0000 7f1ccbfff700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1ccc074040 0x7f1ccc19f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:38846/0 (socket says 172.21.15.131:38846) 2024-02-05T07:50:45.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.933+0000 7f1ccb7fe700 1 -- 172.21.15.131:0/199856573 learned_addr learned my addr 172.21.15.131:0/199856573 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:45.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.934+0000 7f1cd0cf6700 1 --2- 172.21.15.131:0/199856573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ccc10e5c0 0x7f1ccc199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:45.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.934+0000 7f1ccbfff700 1 -- 172.21.15.131:0/199856573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ccc10e5c0 msgr2=0x7f1ccc199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:45.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.934+0000 7f1ccbfff700 1 --2- 172.21.15.131:0/199856573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ccc10e5c0 0x7f1ccc199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.934+0000 7f1ccbfff700 1 -- 172.21.15.131:0/199856573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ccc074960 msgr2=0x7f1ccc19fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.934+0000 7f1ccbfff700 1 --2- 172.21.15.131:0/199856573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ccc074960 0x7f1ccc19fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.934+0000 7f1ccbfff700 1 -- 172.21.15.131:0/199856573 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc00095f0 con 0x7f1ccc074040 2024-02-05T07:50:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.934+0000 7f1ccb7fe700 1 --2- 172.21.15.131:0/199856573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ccc074960 0x7f1ccc19fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:50:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.934+0000 7f1ccbfff700 1 --2- 172.21.15.131:0/199856573 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1ccc074040 0x7f1ccc19f920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009fd0 tx=0x7f1cbc00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.934+0000 7f1cc97fa700 1 -- 172.21.15.131:0/199856573 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc01c070 con 0x7f1ccc074040 2024-02-05T07:50:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.934+0000 7f1cd2759700 1 -- 172.21.15.131:0/199856573 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc009520 con 0x7f1ccc074040 2024-02-05T07:50:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.934+0000 7f1cc97fa700 1 -- 172.21.15.131:0/199856573 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f1cbc00ba10 con 0x7f1ccc074040 2024-02-05T07:50:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.934+0000 7f1cc97fa700 1 -- 172.21.15.131:0/199856573 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0038e0 con 0x7f1ccc074040 2024-02-05T07:50:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.934+0000 7f1cd2759700 1 -- 172.21.15.131:0/199856573 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ccc1a67f0 con 0x7f1ccc074040 2024-02-05T07:50:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.935+0000 7f1cd2759700 1 -- 172.21.15.131:0/199856573 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ccc0689e0 con 0x7f1ccc074040 2024-02-05T07:50:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.936+0000 7f1cc97fa700 1 -- 172.21.15.131:0/199856573 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f1cbc020030 con 0x7f1ccc074040 2024-02-05T07:50:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.936+0000 7f1cc97fa700 1 --2- 172.21.15.131:0/199856573 >> v2:172.21.15.131:6800/2614238408 conn(0x7f1cb406da80 0x7f1cb406ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.936+0000 7f1ccb7fe700 1 --2- 172.21.15.131:0/199856573 >> v2:172.21.15.131:6800/2614238408 conn(0x7f1cb406da80 0x7f1cb406ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.936+0000 7f1cc97fa700 1 -- 172.21.15.131:0/199856573 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f1cbc08e2c0 con 0x7f1ccc074040 2024-02-05T07:50:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.937+0000 7f1ccb7fe700 1 --2- 172.21.15.131:0/199856573 >> v2:172.21.15.131:6800/2614238408 conn(0x7f1cb406da80 0x7f1cb406ff40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0ff540 tx=0x7f1cc0009310 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:45.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:45.941+0000 7f1cc97fa700 1 -- 172.21.15.131:0/199856573 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1cbc05b6e0 con 0x7f1ccc074040 2024-02-05T07:50:46.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:46.140+0000 7f1cd2759700 1 -- 172.21.15.131:0/199856573 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1ccc113750 con 0x7f1ccc074040 2024-02-05T07:50:46.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:46.141+0000 7f1cc97fa700 1 -- 172.21.15.131:0/199856573 <== mon.2 v2:172.21.15.131:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1cbc05b270 con 0x7f1ccc074040 2024-02-05T07:50:46.142 INFO:teuthology.orchestra.run.smithi131.stdout:240518168592 2024-02-05T07:50:46.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:46.143+0000 7f1cd2759700 1 -- 172.21.15.131:0/199856573 >> v2:172.21.15.131:6800/2614238408 conn(0x7f1cb406da80 msgr2=0x7f1cb406ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:46.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:46.143+0000 7f1cd2759700 1 --2- 172.21.15.131:0/199856573 >> v2:172.21.15.131:6800/2614238408 conn(0x7f1cb406da80 0x7f1cb406ff40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0ff540 tx=0x7f1cc0009310 comp rx=0 tx=0).stop 2024-02-05T07:50:46.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:46.143+0000 7f1cd2759700 1 -- 172.21.15.131:0/199856573 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1ccc074040 msgr2=0x7f1ccc19f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:46.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:46.143+0000 7f1cd2759700 1 --2- 172.21.15.131:0/199856573 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1ccc074040 0x7f1ccc19f920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009fd0 tx=0x7f1cbc00fa00 comp rx=0 tx=0).stop 2024-02-05T07:50:46.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:46.143+0000 7f1cd2759700 1 -- 172.21.15.131:0/199856573 shutdown_connections 2024-02-05T07:50:46.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:46.143+0000 7f1cd2759700 1 --2- 172.21.15.131:0/199856573 >> v2:172.21.15.131:6800/2614238408 conn(0x7f1cb406da80 0x7f1cb406ff40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:46.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:46.143+0000 7f1cd2759700 1 --2- 172.21.15.131:0/199856573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ccc10e5c0 0x7f1ccc199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:46.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:46.143+0000 7f1cd2759700 1 --2- 172.21.15.131:0/199856573 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1ccc074040 0x7f1ccc19f920 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:46.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:46.143+0000 7f1cd2759700 1 --2- 172.21.15.131:0/199856573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ccc074960 0x7f1ccc19fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:46.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:46.143+0000 7f1cd2759700 1 -- 172.21.15.131:0/199856573 >> 172.21.15.131:0/199856573 conn(0x7f1ccc0fd7d0 msgr2=0x7f1ccc10f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:46.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:46.143+0000 7f1cd2759700 1 -- 172.21.15.131:0/199856573 shutdown_connections 2024-02-05T07:50:46.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:46.143+0000 7f1cd2759700 1 -- 172.21.15.131:0/199856573 wait complete. 2024-02-05T07:50:47.192 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:46 smithi157 ceph-mon[33345]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:50:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:46 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/199856573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-05T07:50:47.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:46 smithi131 ceph-mon[41654]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:50:47.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:46 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/199856573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-05T07:50:47.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:46 smithi131 ceph-mon[29507]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:50:47.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:46 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/199856573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-05T07:50:47.474 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:47.501 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:50:48.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:50:48 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:50:49.123 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:48 smithi131 ceph-mon[41654]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:50:49.124 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:48 smithi131 ceph-mon[29507]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:50:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:48 smithi157 ceph-mon[33345]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:50:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:49 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:50.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.107+0000 7f271756b700 1 -- 172.21.15.131:0/1739444420 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f2710100c20 msgr2=0x7f2710101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:50.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.107+0000 7f271756b700 1 --2- 172.21.15.131:0/1739444420 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f2710100c20 0x7f2710101080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2700009990 tx=0x7f2700009ca0 comp rx=0 tx=0).stop 2024-02-05T07:50:50.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.107+0000 7f271756b700 1 -- 172.21.15.131:0/1739444420 shutdown_connections 2024-02-05T07:50:50.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.107+0000 7f271756b700 1 --2- 172.21.15.131:0/1739444420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27101015c0 0x7f271010daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:50.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.107+0000 7f271756b700 1 --2- 172.21.15.131:0/1739444420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2710106c60 0x7f2710107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:50.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.107+0000 7f271756b700 1 --2- 172.21.15.131:0/1739444420 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f2710100c20 0x7f2710101080 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:50.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.107+0000 7f271756b700 1 -- 172.21.15.131:0/1739444420 >> 172.21.15.131:0/1739444420 conn(0x7f2710077b50 msgr2=0x7f2710077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:50.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.107+0000 7f271756b700 1 -- 172.21.15.131:0/1739444420 shutdown_connections 2024-02-05T07:50:50.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.107+0000 7f271756b700 1 -- 172.21.15.131:0/1739444420 wait complete. 2024-02-05T07:50:50.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.107+0000 7f271756b700 1 Processor -- start 2024-02-05T07:50:50.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f271756b700 1 -- start start 2024-02-05T07:50:50.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f271756b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2710100c20 0x7f27101028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:50.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f271756b700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27101015c0 0x7f2710102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:50.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f271756b700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f2710106c60 0x7f2710103350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:50.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f271756b700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27101a5e00 con 0x7f27101015c0 2024-02-05T07:50:50.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f271756b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27101a5f70 con 0x7f2710100c20 2024-02-05T07:50:50.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f271756b700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27101a6130 con 0x7f2710106c60 2024-02-05T07:50:50.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f2714b06700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27101015c0 0x7f2710102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:50.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f2714b06700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27101015c0 0x7f2710102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48216/0 (socket says 172.21.15.131:48216) 2024-02-05T07:50:50.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f2715b08700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f2710106c60 0x7f2710103350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:50.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f2714b06700 1 -- 172.21.15.131:0/2464531130 learned_addr learned my addr 172.21.15.131:0/2464531130 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:50.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f2715307700 1 --2- 172.21.15.131:0/2464531130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2710100c20 0x7f27101028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:50.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f2714b06700 1 -- 172.21.15.131:0/2464531130 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f2710106c60 msgr2=0x7f2710103350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:50.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f2714b06700 1 --2- 172.21.15.131:0/2464531130 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f2710106c60 0x7f2710103350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:50.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f2714b06700 1 -- 172.21.15.131:0/2464531130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2710100c20 msgr2=0x7f27101028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:50.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f2714b06700 1 --2- 172.21.15.131:0/2464531130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2710100c20 0x7f27101028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:50.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f2714b06700 1 -- 172.21.15.131:0/2464531130 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2708009c40 con 0x7f27101015c0 2024-02-05T07:50:50.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f2715b08700 1 --2- 172.21.15.131:0/2464531130 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f2710106c60 0x7f2710103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:50:50.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f2714b06700 1 --2- 172.21.15.131:0/2464531130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27101015c0 0x7f2710102e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2700000c00 tx=0x7f270000b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:50.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.108+0000 7f27067fc700 1 -- 172.21.15.131:0/2464531130 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270001d070 con 0x7f27101015c0 2024-02-05T07:50:50.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.109+0000 7f27067fc700 1 -- 172.21.15.131:0/2464531130 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f2700022470 con 0x7f27101015c0 2024-02-05T07:50:50.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.109+0000 7f271756b700 1 -- 172.21.15.131:0/2464531130 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27000095f0 con 0x7f27101015c0 2024-02-05T07:50:50.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.109+0000 7f27067fc700 1 -- 172.21.15.131:0/2464531130 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700017650 con 0x7f27101015c0 2024-02-05T07:50:50.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.109+0000 7f271756b700 1 -- 172.21.15.131:0/2464531130 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27101a6630 con 0x7f27101015c0 2024-02-05T07:50:50.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.110+0000 7f271756b700 1 -- 172.21.15.131:0/2464531130 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27100689e0 con 0x7f27101015c0 2024-02-05T07:50:50.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.110+0000 7f27067fc700 1 -- 172.21.15.131:0/2464531130 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f270000fa50 con 0x7f27101015c0 2024-02-05T07:50:50.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.110+0000 7f27067fc700 1 --2- 172.21.15.131:0/2464531130 >> v2:172.21.15.131:6800/2614238408 conn(0x7f26fc06da80 0x7f26fc06ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:50.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.111+0000 7f2715307700 1 --2- 172.21.15.131:0/2464531130 >> v2:172.21.15.131:6800/2614238408 conn(0x7f26fc06da80 0x7f26fc06ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:50.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.111+0000 7f27067fc700 1 -- 172.21.15.131:0/2464531130 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f270008e8c0 con 0x7f27101015c0 2024-02-05T07:50:50.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.111+0000 7f2715307700 1 --2- 172.21.15.131:0/2464531130 >> v2:172.21.15.131:6800/2614238408 conn(0x7f26fc06da80 0x7f26fc06ff40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f270c006fd0 tx=0x7f270c008040 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:50.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.115+0000 7f27067fc700 1 -- 172.21.15.131:0/2464531130 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f270005be00 con 0x7f27101015c0 2024-02-05T07:50:50.250 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:49 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:50.250 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:49 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:50.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.314+0000 7f271756b700 1 -- 172.21.15.131:0/2464531130 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f27101103f0 con 0x7f27101015c0 2024-02-05T07:50:50.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.315+0000 7f27067fc700 1 -- 172.21.15.131:0/2464531130 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2700027090 con 0x7f27101015c0 2024-02-05T07:50:50.317 INFO:teuthology.orchestra.run.smithi131.stdout:201863462934 2024-02-05T07:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.317+0000 7f271756b700 1 -- 172.21.15.131:0/2464531130 >> v2:172.21.15.131:6800/2614238408 conn(0x7f26fc06da80 msgr2=0x7f26fc06ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.317+0000 7f271756b700 1 --2- 172.21.15.131:0/2464531130 >> v2:172.21.15.131:6800/2614238408 conn(0x7f26fc06da80 0x7f26fc06ff40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f270c006fd0 tx=0x7f270c008040 comp rx=0 tx=0).stop 2024-02-05T07:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.317+0000 7f271756b700 1 -- 172.21.15.131:0/2464531130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27101015c0 msgr2=0x7f2710102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.317+0000 7f271756b700 1 --2- 172.21.15.131:0/2464531130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27101015c0 0x7f2710102e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2700000c00 tx=0x7f270000b680 comp rx=0 tx=0).stop 2024-02-05T07:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.317+0000 7f271756b700 1 -- 172.21.15.131:0/2464531130 shutdown_connections 2024-02-05T07:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.317+0000 7f271756b700 1 --2- 172.21.15.131:0/2464531130 >> v2:172.21.15.131:6800/2614238408 conn(0x7f26fc06da80 0x7f26fc06ff40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.317+0000 7f271756b700 1 --2- 172.21.15.131:0/2464531130 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f2710106c60 0x7f2710103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.317+0000 7f271756b700 1 --2- 172.21.15.131:0/2464531130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2710100c20 0x7f27101028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.317+0000 7f271756b700 1 --2- 172.21.15.131:0/2464531130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27101015c0 0x7f2710102e10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.317+0000 7f271756b700 1 -- 172.21.15.131:0/2464531130 >> 172.21.15.131:0/2464531130 conn(0x7f2710077b50 msgr2=0x7f2710069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.317+0000 7f271756b700 1 -- 172.21.15.131:0/2464531130 shutdown_connections 2024-02-05T07:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:50.317+0000 7f271756b700 1 -- 172.21.15.131:0/2464531130 wait complete. 2024-02-05T07:50:50.886 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168589 got 240518168592 for osd.7 2024-02-05T07:50:50.886 DEBUG:teuthology.parallel:result is None 2024-02-05T07:50:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:50 smithi157 ceph-mon[33345]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2464531130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-05T07:50:51.259 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:50 smithi131 ceph-mon[41654]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:51.259 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2464531130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-05T07:50:51.260 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:50 smithi131 ceph-mon[29507]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:51.260 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2464531130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-05T07:50:52.167 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:51 smithi131 ceph-mon[41654]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:50:52.167 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:51 smithi131 ceph-mon[29507]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:50:52.193 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462932 got 201863462934 for osd.6 2024-02-05T07:50:52.194 DEBUG:teuthology.parallel:result is None 2024-02-05T07:50:52.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:51 smithi157 ceph-mon[33345]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:50:53.062 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:50:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:50:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:50:53.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.649+0000 7f9352eca700 1 -- 172.21.15.131:0/1713372508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f934c101610 msgr2=0x7f934c10dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:53.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.649+0000 7f9352eca700 1 --2- 172.21.15.131:0/1713372508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f934c101610 0x7f934c10dba0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f934400b0d0 tx=0x7f934400b3e0 comp rx=0 tx=0).stop 2024-02-05T07:50:53.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.649+0000 7f9352eca700 1 -- 172.21.15.131:0/1713372508 shutdown_connections 2024-02-05T07:50:53.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.649+0000 7f9352eca700 1 --2- 172.21.15.131:0/1713372508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f934c101610 0x7f934c10dba0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:53.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.649+0000 7f9352eca700 1 --2- 172.21.15.131:0/1713372508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f934c102bb0 0x7f934c102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.649+0000 7f9352eca700 1 --2- 172.21.15.131:0/1713372508 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f934c100c50 0x7f934c1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.649+0000 7f9352eca700 1 -- 172.21.15.131:0/1713372508 >> 172.21.15.131:0/1713372508 conn(0x7f934c076c40 msgr2=0x7f934c077050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.649+0000 7f9352eca700 1 -- 172.21.15.131:0/1713372508 shutdown_connections 2024-02-05T07:50:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.649+0000 7f9352eca700 1 -- 172.21.15.131:0/1713372508 wait complete. 2024-02-05T07:50:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f9352eca700 1 Processor -- start 2024-02-05T07:50:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f9352eca700 1 -- start start 2024-02-05T07:50:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f9352eca700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f934c100c50 0x7f934c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f9352eca700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f934c101610 0x7f934c198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f9352eca700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f934c102bb0 0x7f934c1a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f9352eca700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c199720 con 0x7f934c102bb0 2024-02-05T07:50:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f9352eca700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c1a3f00 con 0x7f934c100c50 2024-02-05T07:50:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f9352eca700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c1a4040 con 0x7f934c101610 2024-02-05T07:50:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f934bfff700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f934c101610 0x7f934c198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f934bfff700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f934c101610 0x7f934c198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:48934/0 (socket says 172.21.15.131:48934) 2024-02-05T07:50:53.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f934bfff700 1 -- 172.21.15.131:0/936821503 learned_addr learned my addr 172.21.15.131:0/936821503 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f9351467700 1 --2- 172.21.15.131:0/936821503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f934c102bb0 0x7f934c1a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f9350c66700 1 --2- 172.21.15.131:0/936821503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f934c100c50 0x7f934c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f934bfff700 1 -- 172.21.15.131:0/936821503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f934c100c50 msgr2=0x7f934c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f934bfff700 1 --2- 172.21.15.131:0/936821503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f934c100c50 0x7f934c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f934bfff700 1 -- 172.21.15.131:0/936821503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f934c102bb0 msgr2=0x7f934c1a39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f934bfff700 1 --2- 172.21.15.131:0/936821503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f934c102bb0 0x7f934c1a39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.650+0000 7f934bfff700 1 -- 172.21.15.131:0/936821503 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9344009d00 con 0x7f934c101610 2024-02-05T07:50:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.651+0000 7f934bfff700 1 --2- 172.21.15.131:0/936821503 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f934c101610 0x7f934c198e80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f933800e9a0 tx=0x7f933800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.651+0000 7f9349ffb700 1 -- 172.21.15.131:0/936821503 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338009790 con 0x7f934c101610 2024-02-05T07:50:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.651+0000 7f9352eca700 1 -- 172.21.15.131:0/936821503 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934c196790 con 0x7f934c101610 2024-02-05T07:50:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.651+0000 7f9349ffb700 1 -- 172.21.15.131:0/936821503 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f9338010430 con 0x7f934c101610 2024-02-05T07:50:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.651+0000 7f9349ffb700 1 -- 172.21.15.131:0/936821503 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338018680 con 0x7f934c101610 2024-02-05T07:50:53.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.651+0000 7f9352eca700 1 -- 172.21.15.131:0/936821503 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934c196c90 con 0x7f934c101610 2024-02-05T07:50:53.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.652+0000 7f9352eca700 1 -- 172.21.15.131:0/936821503 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934c0689e0 con 0x7f934c101610 2024-02-05T07:50:53.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.652+0000 7f9349ffb700 1 -- 172.21.15.131:0/936821503 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f933800ca10 con 0x7f934c101610 2024-02-05T07:50:53.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.653+0000 7f9349ffb700 1 --2- 172.21.15.131:0/936821503 >> v2:172.21.15.131:6800/2614238408 conn(0x7f933c06da80 0x7f933c06ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:53.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.653+0000 7f9349ffb700 1 -- 172.21.15.131:0/936821503 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f9338014070 con 0x7f934c101610 2024-02-05T07:50:53.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.653+0000 7f9350c66700 1 --2- 172.21.15.131:0/936821503 >> v2:172.21.15.131:6800/2614238408 conn(0x7f933c06da80 0x7f933c06ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:53.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.653+0000 7f9350c66700 1 --2- 172.21.15.131:0/936821503 >> v2:172.21.15.131:6800/2614238408 conn(0x7f933c06da80 0x7f933c06ff40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9340005950 tx=0x7f934000b410 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:53.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.657+0000 7f9349ffb700 1 -- 172.21.15.131:0/936821503 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9338052d20 con 0x7f934c101610 2024-02-05T07:50:53.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.859+0000 7f9352eca700 1 -- 172.21.15.131:0/936821503 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f934c1104e0 con 0x7f934c101610 2024-02-05T07:50:53.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.859+0000 7f9349ffb700 1 -- 172.21.15.131:0/936821503 <== mon.2 v2:172.21.15.131:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f933805a980 con 0x7f934c101610 2024-02-05T07:50:53.862 INFO:teuthology.orchestra.run.smithi131.stdout:133143986209 2024-02-05T07:50:53.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.862+0000 7f9352eca700 1 -- 172.21.15.131:0/936821503 >> v2:172.21.15.131:6800/2614238408 conn(0x7f933c06da80 msgr2=0x7f933c06ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:53.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.862+0000 7f9352eca700 1 --2- 172.21.15.131:0/936821503 >> v2:172.21.15.131:6800/2614238408 conn(0x7f933c06da80 0x7f933c06ff40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9340005950 tx=0x7f934000b410 comp rx=0 tx=0).stop 2024-02-05T07:50:53.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.862+0000 7f9352eca700 1 -- 172.21.15.131:0/936821503 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f934c101610 msgr2=0x7f934c198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:53.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.862+0000 7f9352eca700 1 --2- 172.21.15.131:0/936821503 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f934c101610 0x7f934c198e80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f933800e9a0 tx=0x7f933800ecb0 comp rx=0 tx=0).stop 2024-02-05T07:50:53.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.862+0000 7f9352eca700 1 -- 172.21.15.131:0/936821503 shutdown_connections 2024-02-05T07:50:53.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.862+0000 7f9352eca700 1 --2- 172.21.15.131:0/936821503 >> v2:172.21.15.131:6800/2614238408 conn(0x7f933c06da80 0x7f933c06ff40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:53.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.862+0000 7f9352eca700 1 --2- 172.21.15.131:0/936821503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f934c102bb0 0x7f934c1a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:53.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.862+0000 7f9352eca700 1 --2- 172.21.15.131:0/936821503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f934c100c50 0x7f934c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:53.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.862+0000 7f9352eca700 1 --2- 172.21.15.131:0/936821503 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f934c101610 0x7f934c198e80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.862+0000 7f9352eca700 1 -- 172.21.15.131:0/936821503 >> 172.21.15.131:0/936821503 conn(0x7f934c076c40 msgr2=0x7f934c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.862+0000 7f9352eca700 1 -- 172.21.15.131:0/936821503 shutdown_connections 2024-02-05T07:50:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:53.862+0000 7f9352eca700 1 -- 172.21.15.131:0/936821503 wait complete. 2024-02-05T07:50:55.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:54 smithi131 ceph-mon[41654]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:55.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/936821503' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-05T07:50:55.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:54 smithi131 ceph-mon[29507]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:55.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/936821503' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-05T07:50:55.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:54 smithi157 ceph-mon[33345]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:55.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/936821503' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-05T07:50:55.887 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986204 got 133143986209 for osd.4 2024-02-05T07:50:55.887 DEBUG:teuthology.parallel:result is None 2024-02-05T07:50:56.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.355+0000 7f26fede6700 1 -- 172.21.15.131:0/4239003263 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f26f8101860 msgr2=0x7f26f8105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:56.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.355+0000 7f26fede6700 1 --2- 172.21.15.131:0/4239003263 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f26f8101860 0x7f26f8105820 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f26e40098f0 tx=0x7f26e4009c00 comp rx=0 tx=0).stop 2024-02-05T07:50:56.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.355+0000 7f26fede6700 1 -- 172.21.15.131:0/4239003263 shutdown_connections 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.355+0000 7f26fede6700 1 --2- 172.21.15.131:0/4239003263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26f8105f00 0x7f26f810dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.355+0000 7f26fede6700 1 --2- 172.21.15.131:0/4239003263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26f8100eb0 0x7f26f8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.355+0000 7f26fede6700 1 --2- 172.21.15.131:0/4239003263 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f26f8101860 0x7f26f8105820 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.355+0000 7f26fede6700 1 -- 172.21.15.131:0/4239003263 >> 172.21.15.131:0/4239003263 conn(0x7f26f8076c70 msgr2=0x7f26f8077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.355+0000 7f26fede6700 1 -- 172.21.15.131:0/4239003263 shutdown_connections 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.355+0000 7f26fede6700 1 -- 172.21.15.131:0/4239003263 wait complete. 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.356+0000 7f26fede6700 1 Processor -- start 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.356+0000 7f26fede6700 1 -- start start 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.356+0000 7f26fede6700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26f8100eb0 0x7f26f819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.356+0000 7f26fede6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26f8101860 0x7f26f819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.356+0000 7f26fede6700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f26f8105f00 0x7f26f8197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.356+0000 7f26fede6700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f8197ee0 con 0x7f26f8100eb0 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.356+0000 7f26fede6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f8198020 con 0x7f26f8101860 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.356+0000 7f26fede6700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f81981e0 con 0x7f26f8105f00 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.356+0000 7f26fcb82700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26f8100eb0 0x7f26f819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.356+0000 7f26fd383700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f26f8105f00 0x7f26f8197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:56.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.356+0000 7f26fcb82700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26f8100eb0 0x7f26f819d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48262/0 (socket says 172.21.15.131:48262) 2024-02-05T07:50:56.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.356+0000 7f26fcb82700 1 -- 172.21.15.131:0/2748928532 learned_addr learned my addr 172.21.15.131:0/2748928532 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:56.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.356+0000 7f26f7fff700 1 --2- 172.21.15.131:0/2748928532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26f8101860 0x7f26f819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:56.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.357+0000 7f26fd383700 1 -- 172.21.15.131:0/2748928532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26f8101860 msgr2=0x7f26f819dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:56.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.357+0000 7f26fd383700 1 --2- 172.21.15.131:0/2748928532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26f8101860 0x7f26f819dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.357+0000 7f26fd383700 1 -- 172.21.15.131:0/2748928532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26f8100eb0 msgr2=0x7f26f819d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:56.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.357+0000 7f26fd383700 1 --2- 172.21.15.131:0/2748928532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26f8100eb0 0x7f26f819d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.357+0000 7f26fd383700 1 -- 172.21.15.131:0/2748928532 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f0009d00 con 0x7f26f8105f00 2024-02-05T07:50:56.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.357+0000 7f26fcb82700 1 --2- 172.21.15.131:0/2748928532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26f8100eb0 0x7f26f819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:50:56.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.357+0000 7f26fd383700 1 --2- 172.21.15.131:0/2748928532 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f26f8105f00 0x7f26f8197880 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009cd0 tx=0x7f26f000c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:56.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.357+0000 7f26f5ffb700 1 -- 172.21.15.131:0/2748928532 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0017070 con 0x7f26f8105f00 2024-02-05T07:50:56.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.357+0000 7f26f5ffb700 1 -- 172.21.15.131:0/2748928532 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f26f000f040 con 0x7f26f8105f00 2024-02-05T07:50:56.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.357+0000 7f26f5ffb700 1 -- 172.21.15.131:0/2748928532 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f000e050 con 0x7f26f8105f00 2024-02-05T07:50:56.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.357+0000 7f26fede6700 1 -- 172.21.15.131:0/2748928532 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e4009520 con 0x7f26f8105f00 2024-02-05T07:50:56.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.357+0000 7f26fede6700 1 -- 172.21.15.131:0/2748928532 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f81a4590 con 0x7f26f8105f00 2024-02-05T07:50:56.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.358+0000 7f26fede6700 1 -- 172.21.15.131:0/2748928532 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f80689e0 con 0x7f26f8105f00 2024-02-05T07:50:56.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.359+0000 7f26f5ffb700 1 -- 172.21.15.131:0/2748928532 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f26f00090d0 con 0x7f26f8105f00 2024-02-05T07:50:56.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.359+0000 7f26f5ffb700 1 --2- 172.21.15.131:0/2748928532 >> v2:172.21.15.131:6800/2614238408 conn(0x7f26e806da80 0x7f26e806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:56.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.359+0000 7f26f5ffb700 1 -- 172.21.15.131:0/2748928532 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f26f008cdb0 con 0x7f26f8105f00 2024-02-05T07:50:56.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.359+0000 7f26fcb82700 1 --2- 172.21.15.131:0/2748928532 >> v2:172.21.15.131:6800/2614238408 conn(0x7f26e806da80 0x7f26e806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:56.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.359+0000 7f26fcb82700 1 --2- 172.21.15.131:0/2748928532 >> v2:172.21.15.131:6800/2614238408 conn(0x7f26e806da80 0x7f26e806ff40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f26f81989c0 tx=0x7f26ec009290 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:56.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.364+0000 7f26f5ffb700 1 -- 172.21.15.131:0/2748928532 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f26f005a250 con 0x7f26f8105f00 2024-02-05T07:50:56.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.562+0000 7f4d71d50700 1 -- 172.21.15.131:0/3189332084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d6c1015c0 msgr2=0x7f4d6c10daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.562+0000 7f4d71d50700 1 --2- 172.21.15.131:0/3189332084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d6c1015c0 0x7f4d6c10daa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0098f0 tx=0x7f4d5c009c00 comp rx=0 tx=0).stop 2024-02-05T07:50:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.562+0000 7f4d71d50700 1 -- 172.21.15.131:0/3189332084 shutdown_connections 2024-02-05T07:50:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.562+0000 7f4d71d50700 1 --2- 172.21.15.131:0/3189332084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d6c1015c0 0x7f4d6c10daa0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.562+0000 7f4d71d50700 1 --2- 172.21.15.131:0/3189332084 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f4d6c106c60 0x7f4d6c107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.562+0000 7f4d71d50700 1 --2- 172.21.15.131:0/3189332084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d6c100c20 0x7f4d6c101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.562+0000 7f4d71d50700 1 -- 172.21.15.131:0/3189332084 >> 172.21.15.131:0/3189332084 conn(0x7f4d6c077b50 msgr2=0x7f4d6c077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.562+0000 7f4d71d50700 1 -- 172.21.15.131:0/3189332084 shutdown_connections 2024-02-05T07:50:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.562+0000 7f4d71d50700 1 -- 172.21.15.131:0/3189332084 wait complete. 2024-02-05T07:50:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.562+0000 7f4d71d50700 1 Processor -- start 2024-02-05T07:50:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.562+0000 7f4d71d50700 1 -- start start 2024-02-05T07:50:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d71d50700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d6c100c20 0x7f4d6c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:56.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d71d50700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f4d6c1015c0 0x7f4d6c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:56.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d71d50700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d6c106c60 0x7f4d6c196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:56.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d71d50700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c196ee0 con 0x7f4d6c106c60 2024-02-05T07:50:56.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d71d50700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c197050 con 0x7f4d6c100c20 2024-02-05T07:50:56.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d71d50700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c197210 con 0x7f4d6c1015c0 2024-02-05T07:50:56.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d6bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d6c106c60 0x7f4d6c196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:56.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d6affd700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f4d6c1015c0 0x7f4d6c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:56.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d6bfff700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d6c106c60 0x7f4d6c196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48284/0 (socket says 172.21.15.131:48284) 2024-02-05T07:50:56.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d6bfff700 1 -- 172.21.15.131:0/3258037541 learned_addr learned my addr 172.21.15.131:0/3258037541 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:56.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d6b7fe700 1 --2- 172.21.15.131:0/3258037541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d6c100c20 0x7f4d6c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:56.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d6bfff700 1 -- 172.21.15.131:0/3258037541 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f4d6c1015c0 msgr2=0x7f4d6c19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:56.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d6bfff700 1 --2- 172.21.15.131:0/3258037541 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f4d6c1015c0 0x7f4d6c19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d6bfff700 1 -- 172.21.15.131:0/3258037541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d6c100c20 msgr2=0x7f4d6c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:56.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d6bfff700 1 --2- 172.21.15.131:0/3258037541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d6c100c20 0x7f4d6c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d6bfff700 1 -- 172.21.15.131:0/3258037541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c009520 con 0x7f4d6c106c60 2024-02-05T07:50:56.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d6b7fe700 1 --2- 172.21.15.131:0/3258037541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d6c100c20 0x7f4d6c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T07:50:56.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d6bfff700 1 --2- 172.21.15.131:0/3258037541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d6c106c60 0x7f4d6c196730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009810 tx=0x7f4d5c00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:56.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.563+0000 7f4d68ff9700 1 -- 172.21.15.131:0/3258037541 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c01c070 con 0x7f4d6c106c60 2024-02-05T07:50:56.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.564+0000 7f4d71d50700 1 -- 172.21.15.131:0/3258037541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6c1973d0 con 0x7f4d6c106c60 2024-02-05T07:50:56.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.564+0000 7f4d68ff9700 1 -- 172.21.15.131:0/3258037541 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f4d5c00bb90 con 0x7f4d6c106c60 2024-02-05T07:50:56.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.564+0000 7f4d68ff9700 1 -- 172.21.15.131:0/3258037541 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0038c0 con 0x7f4d6c106c60 2024-02-05T07:50:56.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.564+0000 7f4d71d50700 1 -- 172.21.15.131:0/3258037541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6c069ba0 con 0x7f4d6c106c60 2024-02-05T07:50:56.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.564+0000 7f26fede6700 1 -- 172.21.15.131:0/2748928532 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f26f81114e0 con 0x7f26f8105f00 2024-02-05T07:50:56.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.564+0000 7f4d71d50700 1 -- 172.21.15.131:0/3258037541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6c102310 con 0x7f4d6c106c60 2024-02-05T07:50:56.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.565+0000 7f26f5ffb700 1 -- 172.21.15.131:0/2748928532 <== mon.2 v2:172.21.15.131:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f26f0059de0 con 0x7f26f8105f00 2024-02-05T07:50:56.570 INFO:teuthology.orchestra.run.smithi131.stdout:55834574897 2024-02-05T07:50:56.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.565+0000 7f4d68ff9700 1 -- 172.21.15.131:0/3258037541 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f4d5c020030 con 0x7f4d6c106c60 2024-02-05T07:50:56.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.565+0000 7f4d68ff9700 1 --2- 172.21.15.131:0/3258037541 >> v2:172.21.15.131:6800/2614238408 conn(0x7f4d5806da80 0x7f4d5806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:56.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.566+0000 7f4d68ff9700 1 -- 172.21.15.131:0/3258037541 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f4d5c08e370 con 0x7f4d6c106c60 2024-02-05T07:50:56.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.566+0000 7f4d6b7fe700 1 --2- 172.21.15.131:0/3258037541 >> v2:172.21.15.131:6800/2614238408 conn(0x7f4d5806da80 0x7f4d5806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:56.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.566+0000 7f4d6b7fe700 1 --2- 172.21.15.131:0/3258037541 >> v2:172.21.15.131:6800/2614238408 conn(0x7f4d5806da80 0x7f4d5806ff40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4d640093a0 tx=0x7f4d6400b040 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:56.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.567+0000 7f26fede6700 1 -- 172.21.15.131:0/2748928532 >> v2:172.21.15.131:6800/2614238408 conn(0x7f26e806da80 msgr2=0x7f26e806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:56.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.567+0000 7f26fede6700 1 --2- 172.21.15.131:0/2748928532 >> v2:172.21.15.131:6800/2614238408 conn(0x7f26e806da80 0x7f26e806ff40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f26f81989c0 tx=0x7f26ec009290 comp rx=0 tx=0).stop 2024-02-05T07:50:56.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.567+0000 7f26fede6700 1 -- 172.21.15.131:0/2748928532 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f26f8105f00 msgr2=0x7f26f8197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:56.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.567+0000 7f26fede6700 1 --2- 172.21.15.131:0/2748928532 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f26f8105f00 0x7f26f8197880 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009cd0 tx=0x7f26f000c620 comp rx=0 tx=0).stop 2024-02-05T07:50:56.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.567+0000 7f26fede6700 1 -- 172.21.15.131:0/2748928532 shutdown_connections 2024-02-05T07:50:56.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.567+0000 7f26fede6700 1 --2- 172.21.15.131:0/2748928532 >> v2:172.21.15.131:6800/2614238408 conn(0x7f26e806da80 0x7f26e806ff40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.567+0000 7f26fede6700 1 --2- 172.21.15.131:0/2748928532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26f8100eb0 0x7f26f819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.567+0000 7f26fede6700 1 --2- 172.21.15.131:0/2748928532 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f26f8105f00 0x7f26f8197880 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.567+0000 7f26fede6700 1 --2- 172.21.15.131:0/2748928532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26f8101860 0x7f26f819dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.567+0000 7f26fede6700 1 -- 172.21.15.131:0/2748928532 >> 172.21.15.131:0/2748928532 conn(0x7f26f8076c70 msgr2=0x7f26f80feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:56.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.567+0000 7f26fede6700 1 -- 172.21.15.131:0/2748928532 shutdown_connections 2024-02-05T07:50:56.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.567+0000 7f26fede6700 1 -- 172.21.15.131:0/2748928532 wait complete. 2024-02-05T07:50:56.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.571+0000 7f4d68ff9700 1 -- 172.21.15.131:0/3258037541 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4d5c05b790 con 0x7f4d6c106c60 2024-02-05T07:50:56.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.778+0000 7f4d71d50700 1 -- 172.21.15.131:0/3258037541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4d6c197c60 con 0x7f4d6c106c60 2024-02-05T07:50:56.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.778+0000 7f4d68ff9700 1 -- 172.21.15.131:0/3258037541 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4d5c05b320 con 0x7f4d6c106c60 2024-02-05T07:50:56.781 INFO:teuthology.orchestra.run.smithi131.stdout:77309411372 2024-02-05T07:50:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.780+0000 7f4d71d50700 1 -- 172.21.15.131:0/3258037541 >> v2:172.21.15.131:6800/2614238408 conn(0x7f4d5806da80 msgr2=0x7f4d5806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.780+0000 7f4d71d50700 1 --2- 172.21.15.131:0/3258037541 >> v2:172.21.15.131:6800/2614238408 conn(0x7f4d5806da80 0x7f4d5806ff40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4d640093a0 tx=0x7f4d6400b040 comp rx=0 tx=0).stop 2024-02-05T07:50:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.780+0000 7f4d71d50700 1 -- 172.21.15.131:0/3258037541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d6c106c60 msgr2=0x7f4d6c196730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.780+0000 7f4d71d50700 1 --2- 172.21.15.131:0/3258037541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d6c106c60 0x7f4d6c196730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009810 tx=0x7f4d5c00fa70 comp rx=0 tx=0).stop 2024-02-05T07:50:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.781+0000 7f4d71d50700 1 -- 172.21.15.131:0/3258037541 shutdown_connections 2024-02-05T07:50:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.781+0000 7f4d71d50700 1 --2- 172.21.15.131:0/3258037541 >> v2:172.21.15.131:6800/2614238408 conn(0x7f4d5806da80 0x7f4d5806ff40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.781+0000 7f4d71d50700 1 --2- 172.21.15.131:0/3258037541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d6c106c60 0x7f4d6c196730 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.781+0000 7f4d71d50700 1 --2- 172.21.15.131:0/3258037541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d6c100c20 0x7f4d6c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.781+0000 7f4d71d50700 1 --2- 172.21.15.131:0/3258037541 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f4d6c1015c0 0x7f4d6c19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.781+0000 7f4d71d50700 1 -- 172.21.15.131:0/3258037541 >> 172.21.15.131:0/3258037541 conn(0x7f4d6c077b50 msgr2=0x7f4d6c0ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.781+0000 7f4d71d50700 1 -- 172.21.15.131:0/3258037541 shutdown_connections 2024-02-05T07:50:56.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:56.781+0000 7f4d71d50700 1 -- 172.21.15.131:0/3258037541 wait complete. 2024-02-05T07:50:57.206 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:56 smithi131 ceph-mon[41654]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:57.206 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2748928532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-05T07:50:57.206 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3258037541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-05T07:50:57.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:56 smithi131 ceph-mon[29507]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:57.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2748928532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-05T07:50:57.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3258037541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-05T07:50:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:56 smithi157 ceph-mon[33345]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2748928532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-05T07:50:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3258037541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-05T07:50:57.233 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574894 got 55834574897 for osd.1 2024-02-05T07:50:57.233 DEBUG:teuthology.parallel:result is None 2024-02-05T07:50:57.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.626+0000 7fc207b44700 1 -- 172.21.15.131:0/2502645329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200102730 msgr2=0x7fc20010eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:57.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.626+0000 7fc207b44700 1 --2- 172.21.15.131:0/2502645329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200102730 0x7fc20010eb70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009fc0 tx=0x7fc1f800b330 comp rx=0 tx=0).stop 2024-02-05T07:50:57.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.626+0000 7fc207b44700 1 -- 172.21.15.131:0/2502645329 shutdown_connections 2024-02-05T07:50:57.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.626+0000 7fc207b44700 1 --2- 172.21.15.131:0/2502645329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200102730 0x7fc20010eb70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:57.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.626+0000 7fc207b44700 1 --2- 172.21.15.131:0/2502645329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc200107d50 0x7fc200108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:57.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.626+0000 7fc207b44700 1 --2- 172.21.15.131:0/2502645329 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc200101d70 0x7fc2001021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:57.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.626+0000 7fc207b44700 1 -- 172.21.15.131:0/2502645329 >> 172.21.15.131:0/2502645329 conn(0x7fc200078e50 msgr2=0x7fc200079260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:57.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.627+0000 7fc207b44700 1 -- 172.21.15.131:0/2502645329 shutdown_connections 2024-02-05T07:50:57.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.627+0000 7fc207b44700 1 -- 172.21.15.131:0/2502645329 wait complete. 2024-02-05T07:50:57.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.627+0000 7fc207b44700 1 Processor -- start 2024-02-05T07:50:57.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.627+0000 7fc207b44700 1 -- start start 2024-02-05T07:50:57.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.627+0000 7fc207b44700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc200101d70 0x7fc20019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:57.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.627+0000 7fc207b44700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc200102730 0x7fc20019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:57.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.627+0000 7fc207b44700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200107d50 0x7fc200199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:57.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.627+0000 7fc207b44700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20019a0a0 con 0x7fc200107d50 2024-02-05T07:50:57.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.627+0000 7fc207b44700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20019a1e0 con 0x7fc200102730 2024-02-05T07:50:57.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.627+0000 7fc207b44700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20019a3a0 con 0x7fc200101d70 2024-02-05T07:50:57.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.627+0000 7fc2058e0700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc200101d70 0x7fc20019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:57.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.627+0000 7fc2060e1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200107d50 0x7fc200199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:57.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.627+0000 7fc2060e1700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200107d50 0x7fc200199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48298/0 (socket says 172.21.15.131:48298) 2024-02-05T07:50:57.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.627+0000 7fc2058e0700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc200101d70 0x7fc20019f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:48978/0 (socket says 172.21.15.131:48978) 2024-02-05T07:50:57.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.627+0000 7fc2060e1700 1 -- 172.21.15.131:0/4063305290 learned_addr learned my addr 172.21.15.131:0/4063305290 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:57.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.628+0000 7fc2050df700 1 --2- 172.21.15.131:0/4063305290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc200102730 0x7fc20019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:57.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.628+0000 7fc2060e1700 1 -- 172.21.15.131:0/4063305290 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc200101d70 msgr2=0x7fc20019f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:57.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.628+0000 7fc2060e1700 1 --2- 172.21.15.131:0/4063305290 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc200101d70 0x7fc20019f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:57.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.628+0000 7fc2060e1700 1 -- 172.21.15.131:0/4063305290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc200102730 msgr2=0x7fc20019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:57.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.628+0000 7fc2060e1700 1 --2- 172.21.15.131:0/4063305290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc200102730 0x7fc20019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:57.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.628+0000 7fc2060e1700 1 -- 172.21.15.131:0/4063305290 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f00095f0 con 0x7fc200107d50 2024-02-05T07:50:57.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.628+0000 7fc2058e0700 1 --2- 172.21.15.131:0/4063305290 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc200101d70 0x7fc20019f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:50:57.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.628+0000 7fc2060e1700 1 --2- 172.21.15.131:0/4063305290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200107d50 0x7fc200199a40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009450 tx=0x7fc1f800b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:57.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.628+0000 7fc1f6ffd700 1 -- 172.21.15.131:0/4063305290 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800e040 con 0x7fc200107d50 2024-02-05T07:50:57.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.628+0000 7fc207b44700 1 -- 172.21.15.131:0/4063305290 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f8009c40 con 0x7fc200107d50 2024-02-05T07:50:57.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.628+0000 7fc1f6ffd700 1 -- 172.21.15.131:0/4063305290 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc1f8007700 con 0x7fc200107d50 2024-02-05T07:50:57.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.628+0000 7fc1f6ffd700 1 -- 172.21.15.131:0/4063305290 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f80052a0 con 0x7fc200107d50 2024-02-05T07:50:57.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.628+0000 7fc207b44700 1 -- 172.21.15.131:0/4063305290 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2001a6750 con 0x7fc200107d50 2024-02-05T07:50:57.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.629+0000 7fc207b44700 1 -- 172.21.15.131:0/4063305290 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2000689e0 con 0x7fc200107d50 2024-02-05T07:50:57.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.630+0000 7fc1f6ffd700 1 -- 172.21.15.131:0/4063305290 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fc1f801d470 con 0x7fc200107d50 2024-02-05T07:50:57.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.630+0000 7fc1f6ffd700 1 --2- 172.21.15.131:0/4063305290 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc1ec06da80 0x7fc1ec06ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:57.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.630+0000 7fc2058e0700 1 --2- 172.21.15.131:0/4063305290 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc1ec06da80 0x7fc1ec06ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:57.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.630+0000 7fc1f6ffd700 1 -- 172.21.15.131:0/4063305290 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fc1f808e830 con 0x7fc200107d50 2024-02-05T07:50:57.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.630+0000 7fc2058e0700 1 --2- 172.21.15.131:0/4063305290 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc1ec06da80 0x7fc1ec06ff40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc005950 tx=0x7fc1fc00a300 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:57.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.635+0000 7fc1f6ffd700 1 -- 172.21.15.131:0/4063305290 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1f805bc50 con 0x7fc200107d50 2024-02-05T07:50:57.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.839+0000 7fc207b44700 1 -- 172.21.15.131:0/4063305290 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc2001114e0 con 0x7fc200107d50 2024-02-05T07:50:57.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.839+0000 7fc1f6ffd700 1 -- 172.21.15.131:0/4063305290 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc1f805b7e0 con 0x7fc200107d50 2024-02-05T07:50:57.842 INFO:teuthology.orchestra.run.smithi131.stdout:111669149735 2024-02-05T07:50:57.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.842+0000 7fc207b44700 1 -- 172.21.15.131:0/4063305290 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc1ec06da80 msgr2=0x7fc1ec06ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:57.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.842+0000 7fc207b44700 1 --2- 172.21.15.131:0/4063305290 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc1ec06da80 0x7fc1ec06ff40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc005950 tx=0x7fc1fc00a300 comp rx=0 tx=0).stop 2024-02-05T07:50:57.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.842+0000 7fc207b44700 1 -- 172.21.15.131:0/4063305290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200107d50 msgr2=0x7fc200199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:57.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.842+0000 7fc207b44700 1 --2- 172.21.15.131:0/4063305290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200107d50 0x7fc200199a40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009450 tx=0x7fc1f800b9f0 comp rx=0 tx=0).stop 2024-02-05T07:50:57.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.842+0000 7fc207b44700 1 -- 172.21.15.131:0/4063305290 shutdown_connections 2024-02-05T07:50:57.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.842+0000 7fc207b44700 1 --2- 172.21.15.131:0/4063305290 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc1ec06da80 0x7fc1ec06ff40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:57.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.842+0000 7fc207b44700 1 --2- 172.21.15.131:0/4063305290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc200107d50 0x7fc200199a40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:57.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.842+0000 7fc207b44700 1 --2- 172.21.15.131:0/4063305290 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc200101d70 0x7fc20019f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:57.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.842+0000 7fc207b44700 1 --2- 172.21.15.131:0/4063305290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc200102730 0x7fc20019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:57.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.842+0000 7fc207b44700 1 -- 172.21.15.131:0/4063305290 >> 172.21.15.131:0/4063305290 conn(0x7fc200078e50 msgr2=0x7fc20010cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:57.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.842+0000 7fc207b44700 1 -- 172.21.15.131:0/4063305290 shutdown_connections 2024-02-05T07:50:57.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.842+0000 7fc207b44700 1 -- 172.21.15.131:0/4063305290 wait complete. 2024-02-05T07:50:57.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.830+0000 7fdf2e5b8700 1 -- 172.21.15.131:0/283967139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf28101860 msgr2=0x7fdf28105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:57.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.830+0000 7fdf2e5b8700 1 --2- 172.21.15.131:0/283967139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf28101860 0x7fdf28105820 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdf180098f0 tx=0x7fdf18009c00 comp rx=0 tx=0).stop 2024-02-05T07:50:57.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.831+0000 7fdf2e5b8700 1 -- 172.21.15.131:0/283967139 shutdown_connections 2024-02-05T07:50:57.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.831+0000 7fdf2e5b8700 1 --2- 172.21.15.131:0/283967139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf28105f00 0x7fdf2810dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:57.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.831+0000 7fdf2e5b8700 1 --2- 172.21.15.131:0/283967139 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fdf28100eb0 0x7fdf28101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:57.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.831+0000 7fdf2e5b8700 1 --2- 172.21.15.131:0/283967139 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf28101860 0x7fdf28105820 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:57.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.831+0000 7fdf2e5b8700 1 -- 172.21.15.131:0/283967139 >> 172.21.15.131:0/283967139 conn(0x7fdf28076c70 msgr2=0x7fdf28077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:57.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.831+0000 7fdf2e5b8700 1 -- 172.21.15.131:0/283967139 shutdown_connections 2024-02-05T07:50:57.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.831+0000 7fdf2e5b8700 1 -- 172.21.15.131:0/283967139 wait complete. 2024-02-05T07:50:57.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.831+0000 7fdf2e5b8700 1 Processor -- start 2024-02-05T07:50:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.831+0000 7fdf2e5b8700 1 -- start start 2024-02-05T07:50:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.832+0000 7fdf2e5b8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf28100eb0 0x7fdf281988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.832+0000 7fdf2e5b8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf28101860 0x7fdf28198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.832+0000 7fdf2e5b8700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fdf28105f00 0x7fdf281a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.832+0000 7fdf2e5b8700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28199640 con 0x7fdf28101860 2024-02-05T07:50:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.832+0000 7fdf2e5b8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf281a3e70 con 0x7fdf28100eb0 2024-02-05T07:50:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.832+0000 7fdf2e5b8700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf281a3fb0 con 0x7fdf28105f00 2024-02-05T07:50:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.832+0000 7fdf2cb55700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fdf28105f00 0x7fdf281a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.832+0000 7fdf2cb55700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fdf28105f00 0x7fdf281a3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:48994/0 (socket says 172.21.15.131:48994) 2024-02-05T07:50:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.832+0000 7fdf2cb55700 1 -- 172.21.15.131:0/98110250 learned_addr learned my addr 172.21.15.131:0/98110250 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:50:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.832+0000 7fdf2cb55700 1 -- 172.21.15.131:0/98110250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf28100eb0 msgr2=0x7fdf281988b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-05T07:50:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.832+0000 7fdf2cb55700 1 --2- 172.21.15.131:0/98110250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf28100eb0 0x7fdf281988b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.832+0000 7fdf2cb55700 1 -- 172.21.15.131:0/98110250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf28101860 msgr2=0x7fdf28198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.832+0000 7fdf2cb55700 1 --2- 172.21.15.131:0/98110250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf28101860 0x7fdf28198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:57.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.832+0000 7fdf2cb55700 1 -- 172.21.15.131:0/98110250 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c0095f0 con 0x7fdf28105f00 2024-02-05T07:50:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.833+0000 7fdf2cb55700 1 --2- 172.21.15.131:0/98110250 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fdf28105f00 0x7fdf281a3930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000b970 tx=0x7fdf2000bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.833+0000 7fdf257fa700 1 -- 172.21.15.131:0/98110250 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2000c9b0 con 0x7fdf28105f00 2024-02-05T07:50:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.833+0000 7fdf257fa700 1 -- 172.21.15.131:0/98110250 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fdf20014440 con 0x7fdf28105f00 2024-02-05T07:50:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.833+0000 7fdf2e5b8700 1 -- 172.21.15.131:0/98110250 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf18009520 con 0x7fdf28105f00 2024-02-05T07:50:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.833+0000 7fdf257fa700 1 -- 172.21.15.131:0/98110250 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf200075d0 con 0x7fdf28105f00 2024-02-05T07:50:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.833+0000 7fdf2e5b8700 1 -- 172.21.15.131:0/98110250 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf281a4530 con 0x7fdf28105f00 2024-02-05T07:50:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.834+0000 7fdf2e5b8700 1 -- 172.21.15.131:0/98110250 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf280689e0 con 0x7fdf28105f00 2024-02-05T07:50:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.835+0000 7fdf257fa700 1 -- 172.21.15.131:0/98110250 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fdf20007730 con 0x7fdf28105f00 2024-02-05T07:50:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.835+0000 7fdf257fa700 1 --2- 172.21.15.131:0/98110250 >> v2:172.21.15.131:6800/2614238408 conn(0x7fdf1006da80 0x7fdf1006ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:50:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.835+0000 7fdf257fa700 1 -- 172.21.15.131:0/98110250 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fdf20006e40 con 0x7fdf28105f00 2024-02-05T07:50:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.835+0000 7fdf27fff700 1 --2- 172.21.15.131:0/98110250 >> v2:172.21.15.131:6800/2614238408 conn(0x7fdf1006da80 0x7fdf1006ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:50:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.836+0000 7fdf27fff700 1 --2- 172.21.15.131:0/98110250 >> v2:172.21.15.131:6800/2614238408 conn(0x7fdf1006da80 0x7fdf1006ff40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005fd0 tx=0x7fdf1c009310 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:50:57.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:57.840+0000 7fdf257fa700 1 -- 172.21.15.131:0/98110250 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf2005a000 con 0x7fdf28105f00 2024-02-05T07:50:58.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:58.047+0000 7fdf2e5b8700 1 -- 172.21.15.131:0/98110250 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fdf281103c0 con 0x7fdf28105f00 2024-02-05T07:50:58.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:58.048+0000 7fdf257fa700 1 -- 172.21.15.131:0/98110250 <== mon.2 v2:172.21.15.131:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf20059b90 con 0x7fdf28105f00 2024-02-05T07:50:58.050 INFO:teuthology.orchestra.run.smithi131.stdout:167503724573 2024-02-05T07:50:58.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:58.050+0000 7fdf2e5b8700 1 -- 172.21.15.131:0/98110250 >> v2:172.21.15.131:6800/2614238408 conn(0x7fdf1006da80 msgr2=0x7fdf1006ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:58.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:58.050+0000 7fdf2e5b8700 1 --2- 172.21.15.131:0/98110250 >> v2:172.21.15.131:6800/2614238408 conn(0x7fdf1006da80 0x7fdf1006ff40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005fd0 tx=0x7fdf1c009310 comp rx=0 tx=0).stop 2024-02-05T07:50:58.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:58.050+0000 7fdf2e5b8700 1 -- 172.21.15.131:0/98110250 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fdf28105f00 msgr2=0x7fdf281a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:50:58.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:58.050+0000 7fdf2e5b8700 1 --2- 172.21.15.131:0/98110250 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fdf28105f00 0x7fdf281a3930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000b970 tx=0x7fdf2000bd30 comp rx=0 tx=0).stop 2024-02-05T07:50:58.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:58.050+0000 7fdf2e5b8700 1 -- 172.21.15.131:0/98110250 shutdown_connections 2024-02-05T07:50:58.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:58.050+0000 7fdf2e5b8700 1 --2- 172.21.15.131:0/98110250 >> v2:172.21.15.131:6800/2614238408 conn(0x7fdf1006da80 0x7fdf1006ff40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:58.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:58.050+0000 7fdf2e5b8700 1 --2- 172.21.15.131:0/98110250 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fdf28105f00 0x7fdf281a3930 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:58.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:58.050+0000 7fdf2e5b8700 1 --2- 172.21.15.131:0/98110250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf28100eb0 0x7fdf281988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:58.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:58.050+0000 7fdf2e5b8700 1 --2- 172.21.15.131:0/98110250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf28101860 0x7fdf28198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:50:58.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:58.050+0000 7fdf2e5b8700 1 -- 172.21.15.131:0/98110250 >> 172.21.15.131:0/98110250 conn(0x7fdf28076c70 msgr2=0x7fdf280feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:50:58.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:58.050+0000 7fdf2e5b8700 1 -- 172.21.15.131:0/98110250 shutdown_connections 2024-02-05T07:50:58.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:50:58.050+0000 7fdf2e5b8700 1 -- 172.21.15.131:0/98110250 wait complete. 2024-02-05T07:50:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:57 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:57 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4063305290' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-05T07:50:58.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:57 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:58.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:57 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4063305290' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-05T07:50:58.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:57 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:58.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:57 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4063305290' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-05T07:50:58.627 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724570 got 167503724573 for osd.5 2024-02-05T07:50:58.628 DEBUG:teuthology.parallel:result is None 2024-02-05T07:50:58.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:50:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:50:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:58 smithi131 ceph-mon[41654]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:50:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/98110250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-05T07:50:59.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:58 smithi131 ceph-mon[29507]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:50:59.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/98110250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-05T07:50:59.093 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149731 got 111669149735 for osd.3 2024-02-05T07:50:59.094 DEBUG:teuthology.parallel:result is None 2024-02-05T07:50:59.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:58 smithi157 ceph-mon[33345]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:50:59.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/98110250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-05T07:50:59.935 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411369 got 77309411372 for osd.2 2024-02-05T07:50:59.935 DEBUG:teuthology.parallel:result is None 2024-02-05T07:51:00.179 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:59 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:00.179 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:59 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:00.179 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:50:59 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:00.180 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:59 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:00.180 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:59 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:00.180 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:50:59 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:59 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:59 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:50:59 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:00.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.461+0000 7f1208a02700 1 -- 172.21.15.131:0/573588671 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1204102e20 msgr2=0x7f1204103200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.461+0000 7f1208a02700 1 --2- 172.21.15.131:0/573588671 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1204102e20 0x7f1204103200 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f11f40098f0 tx=0x7f11f4009c00 comp rx=0 tx=0).stop 2024-02-05T07:51:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.461+0000 7f1208a02700 1 -- 172.21.15.131:0/573588671 shutdown_connections 2024-02-05T07:51:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.461+0000 7f1208a02700 1 --2- 172.21.15.131:0/573588671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f120410c200 0x7f120410e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.461+0000 7f1208a02700 1 --2- 172.21.15.131:0/573588671 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1204102e20 0x7f1204103200 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.461+0000 7f1208a02700 1 --2- 172.21.15.131:0/573588671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1204103740 0x7f120410bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.461+0000 7f1208a02700 1 -- 172.21.15.131:0/573588671 >> 172.21.15.131:0/573588671 conn(0x7f12040fc5f0 msgr2=0x7f12040fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:51:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.461+0000 7f1208a02700 1 -- 172.21.15.131:0/573588671 shutdown_connections 2024-02-05T07:51:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.461+0000 7f1208a02700 1 -- 172.21.15.131:0/573588671 wait complete. 2024-02-05T07:51:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.462+0000 7f1208a02700 1 Processor -- start 2024-02-05T07:51:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.462+0000 7f1208a02700 1 -- start start 2024-02-05T07:51:00.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.462+0000 7f1208a02700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1204102e20 0x7f120419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:51:00.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.462+0000 7f1208a02700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1204103740 0x7f120419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:51:00.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.462+0000 7f1208a02700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f120410c200 0x7f1204197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:51:00.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.462+0000 7f1208a02700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1204197ed0 con 0x7f120410c200 2024-02-05T07:51:00.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.462+0000 7f1208a02700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1204198040 con 0x7f1204103740 2024-02-05T07:51:00.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.462+0000 7f1208a02700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1204198200 con 0x7f1204102e20 2024-02-05T07:51:00.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.462+0000 7f120259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1204103740 0x7f120419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:51:00.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.462+0000 7f1202d9d700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1204102e20 0x7f120419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:51:00.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.462+0000 7f120259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1204103740 0x7f120419db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:56170/0 (socket says 172.21.15.131:56170) 2024-02-05T07:51:00.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.462+0000 7f120259c700 1 -- 172.21.15.131:0/4251805892 learned_addr learned my addr 172.21.15.131:0/4251805892 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:51:00.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.462+0000 7f120359e700 1 --2- 172.21.15.131:0/4251805892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f120410c200 0x7f1204197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:51:00.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.463+0000 7f120259c700 1 -- 172.21.15.131:0/4251805892 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1204102e20 msgr2=0x7f120419d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:00.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.463+0000 7f120259c700 1 --2- 172.21.15.131:0/4251805892 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1204102e20 0x7f120419d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:00.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.463+0000 7f120259c700 1 -- 172.21.15.131:0/4251805892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f120410c200 msgr2=0x7f1204197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:00.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.463+0000 7f120259c700 1 --2- 172.21.15.131:0/4251805892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f120410c200 0x7f1204197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:00.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.463+0000 7f120259c700 1 -- 172.21.15.131:0/4251805892 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f80095f0 con 0x7f1204103740 2024-02-05T07:51:00.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.463+0000 7f120259c700 1 --2- 172.21.15.131:0/4251805892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1204103740 0x7f120419db90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f11f800ec70 tx=0x7f11f800ef80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:51:00.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.463+0000 7f11ebfff700 1 -- 172.21.15.131:0/4251805892 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f80099d0 con 0x7f1204103740 2024-02-05T07:51:00.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.463+0000 7f1208a02700 1 -- 172.21.15.131:0/4251805892 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f4009520 con 0x7f1204103740 2024-02-05T07:51:00.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.463+0000 7f11ebfff700 1 -- 172.21.15.131:0/4251805892 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f11f80044d0 con 0x7f1204103740 2024-02-05T07:51:00.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.463+0000 7f1208a02700 1 -- 172.21.15.131:0/4251805892 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12041a45d0 con 0x7f1204103740 2024-02-05T07:51:00.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.463+0000 7f11ebfff700 1 -- 172.21.15.131:0/4251805892 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f8003e70 con 0x7f1204103740 2024-02-05T07:51:00.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.464+0000 7f1208a02700 1 -- 172.21.15.131:0/4251805892 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1204107380 con 0x7f1204103740 2024-02-05T07:51:00.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.465+0000 7f11ebfff700 1 -- 172.21.15.131:0/4251805892 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f11f8004000 con 0x7f1204103740 2024-02-05T07:51:00.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.465+0000 7f11ebfff700 1 --2- 172.21.15.131:0/4251805892 >> v2:172.21.15.131:6800/2614238408 conn(0x7f11ec06da80 0x7f11ec06ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:51:00.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.465+0000 7f11ebfff700 1 -- 172.21.15.131:0/4251805892 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7f11f8014070 con 0x7f1204103740 2024-02-05T07:51:00.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.465+0000 7f1202d9d700 1 --2- 172.21.15.131:0/4251805892 >> v2:172.21.15.131:6800/2614238408 conn(0x7f11ec06da80 0x7f11ec06ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:51:00.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.466+0000 7f1202d9d700 1 --2- 172.21.15.131:0/4251805892 >> v2:172.21.15.131:6800/2614238408 conn(0x7f11ec06da80 0x7f11ec06ff40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f11f40098c0 tx=0x7f11f4019040 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:51:00.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.470+0000 7f11ebfff700 1 -- 172.21.15.131:0/4251805892 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f11f805abe0 con 0x7f1204103740 2024-02-05T07:51:00.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.742+0000 7f1208a02700 1 -- 172.21.15.131:0/4251805892 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f12041983c0 con 0x7f1204103740 2024-02-05T07:51:00.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.742+0000 7f11ebfff700 1 -- 172.21.15.131:0/4251805892 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f11f805a770 con 0x7f1204103740 2024-02-05T07:51:00.744 INFO:teuthology.orchestra.run.smithi131.stdout:34359738423 2024-02-05T07:51:00.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.745+0000 7f1208a02700 1 -- 172.21.15.131:0/4251805892 >> v2:172.21.15.131:6800/2614238408 conn(0x7f11ec06da80 msgr2=0x7f11ec06ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:00.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.745+0000 7f1208a02700 1 --2- 172.21.15.131:0/4251805892 >> v2:172.21.15.131:6800/2614238408 conn(0x7f11ec06da80 0x7f11ec06ff40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f11f40098c0 tx=0x7f11f4019040 comp rx=0 tx=0).stop 2024-02-05T07:51:00.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.745+0000 7f1208a02700 1 -- 172.21.15.131:0/4251805892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1204103740 msgr2=0x7f120419db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:00.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.745+0000 7f1208a02700 1 --2- 172.21.15.131:0/4251805892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1204103740 0x7f120419db90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f11f800ec70 tx=0x7f11f800ef80 comp rx=0 tx=0).stop 2024-02-05T07:51:00.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.745+0000 7f1208a02700 1 -- 172.21.15.131:0/4251805892 shutdown_connections 2024-02-05T07:51:00.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.745+0000 7f1208a02700 1 --2- 172.21.15.131:0/4251805892 >> v2:172.21.15.131:6800/2614238408 conn(0x7f11ec06da80 0x7f11ec06ff40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:00.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.745+0000 7f1208a02700 1 --2- 172.21.15.131:0/4251805892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f120410c200 0x7f1204197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:00.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.745+0000 7f1208a02700 1 --2- 172.21.15.131:0/4251805892 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f1204102e20 0x7f120419d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:00.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.745+0000 7f1208a02700 1 --2- 172.21.15.131:0/4251805892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1204103740 0x7f120419db90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:00.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.745+0000 7f1208a02700 1 -- 172.21.15.131:0/4251805892 >> 172.21.15.131:0/4251805892 conn(0x7f12040fc5f0 msgr2=0x7f12040fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:51:00.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.745+0000 7f1208a02700 1 -- 172.21.15.131:0/4251805892 shutdown_connections 2024-02-05T07:51:00.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:00.745+0000 7f1208a02700 1 -- 172.21.15.131:0/4251805892 wait complete. 2024-02-05T07:51:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[41654]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:01.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:51:01.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:51:01.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4251805892' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-05T07:51:01.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[29507]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:01.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:01.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:01.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:01.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:51:01.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:51:01.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4251805892' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-05T07:51:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:00 smithi157 ceph-mon[33345]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:00 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:00 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:00 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:00 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:00 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:00 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:51:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:00 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:01.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:51:01.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:51:01.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4251805892' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-05T07:51:01.508 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738418 got 34359738423 for osd.0 2024-02-05T07:51:01.508 DEBUG:teuthology.parallel:result is None 2024-02-05T07:51:01.508 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-05T07:51:01.508 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph pg dump --format=json 2024-02-05T07:51:01.753 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:51:02.173 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[41654]: Adjusting osd_memory_target on smithi131 to 3760M 2024-02-05T07:51:02.173 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[41654]: Updating smithi131:/etc/ceph/ceph.conf 2024-02-05T07:51:02.173 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[41654]: Updating smithi157:/etc/ceph/ceph.conf 2024-02-05T07:51:02.173 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[41654]: Updating smithi131:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:51:02.173 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[41654]: Updating smithi157:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:51:02.173 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[41654]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:51:02.174 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[41654]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:51:02.174 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:02.174 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:02.174 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:02.174 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:02.174 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:02.174 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[29507]: Adjusting osd_memory_target on smithi131 to 3760M 2024-02-05T07:51:02.174 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[29507]: Updating smithi131:/etc/ceph/ceph.conf 2024-02-05T07:51:02.174 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[29507]: Updating smithi157:/etc/ceph/ceph.conf 2024-02-05T07:51:02.174 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[29507]: Updating smithi131:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:51:02.174 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[29507]: Updating smithi157:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:51:02.174 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[29507]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:51:02.174 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[29507]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:51:02.174 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:02.174 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:02.175 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:02.175 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:02.175 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:01 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:01 smithi157 ceph-mon[33345]: Adjusting osd_memory_target on smithi131 to 3760M 2024-02-05T07:51:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:01 smithi157 ceph-mon[33345]: Updating smithi131:/etc/ceph/ceph.conf 2024-02-05T07:51:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:01 smithi157 ceph-mon[33345]: Updating smithi157:/etc/ceph/ceph.conf 2024-02-05T07:51:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:01 smithi157 ceph-mon[33345]: Updating smithi131:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:51:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:01 smithi157 ceph-mon[33345]: Updating smithi157:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:51:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:01 smithi157 ceph-mon[33345]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:51:02.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:01 smithi157 ceph-mon[33345]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:51:02.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:01 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:02.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:01 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:02.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:01 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:02.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:01 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:02.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:01 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:02.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:02 smithi131 ceph-mon[41654]: Updating smithi131:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:51:02.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:02 smithi131 ceph-mon[41654]: Updating smithi157:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:51:02.944 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:02 smithi131 ceph-mon[41654]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:02.944 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:02 smithi131 ceph-mon[41654]: Deploying daemon alertmanager.a on smithi131 2024-02-05T07:51:02.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:02 smithi131 ceph-mon[29507]: Updating smithi131:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:51:02.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:02 smithi131 ceph-mon[29507]: Updating smithi157:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:51:02.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:02 smithi131 ceph-mon[29507]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:02.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:02 smithi131 ceph-mon[29507]: Deploying daemon alertmanager.a on smithi131 2024-02-05T07:51:03.195 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:51:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:51:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:02 smithi157 ceph-mon[33345]: Updating smithi131:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:51:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:02 smithi157 ceph-mon[33345]: Updating smithi157:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:51:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:02 smithi157 ceph-mon[33345]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:02 smithi157 ceph-mon[33345]: Deploying daemon alertmanager.a on smithi131 2024-02-05T07:51:03.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.639+0000 7fa8cc03c700 1 -- 172.21.15.131:0/1708106027 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa8c41015c0 msgr2=0x7fa8c410daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:03.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.639+0000 7fa8cc03c700 1 --2- 172.21.15.131:0/1708106027 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa8c41015c0 0x7fa8c410daa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00b0a0 tx=0x7fa8bc00b3b0 comp rx=0 tx=0).stop 2024-02-05T07:51:03.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.639+0000 7fa8cc03c700 1 -- 172.21.15.131:0/1708106027 shutdown_connections 2024-02-05T07:51:03.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.639+0000 7fa8cc03c700 1 --2- 172.21.15.131:0/1708106027 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa8c41015c0 0x7fa8c410daa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:03.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.639+0000 7fa8cc03c700 1 --2- 172.21.15.131:0/1708106027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8c4106c60 0x7fa8c4107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:03.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.639+0000 7fa8cc03c700 1 --2- 172.21.15.131:0/1708106027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8c4100c20 0x7fa8c4101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:03.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.639+0000 7fa8cc03c700 1 -- 172.21.15.131:0/1708106027 >> 172.21.15.131:0/1708106027 conn(0x7fa8c4077b50 msgr2=0x7fa8c4077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:51:03.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.639+0000 7fa8cc03c700 1 -- 172.21.15.131:0/1708106027 shutdown_connections 2024-02-05T07:51:03.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.639+0000 7fa8cc03c700 1 -- 172.21.15.131:0/1708106027 wait complete. 2024-02-05T07:51:03.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.640+0000 7fa8cc03c700 1 Processor -- start 2024-02-05T07:51:03.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.640+0000 7fa8cc03c700 1 -- start start 2024-02-05T07:51:03.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.640+0000 7fa8cc03c700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8c4100c20 0x7fa8c4196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:51:03.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.640+0000 7fa8cc03c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8c41015c0 0x7fa8c4196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:51:03.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.640+0000 7fa8cc03c700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa8c4106c60 0x7fa8c41970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:51:03.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.640+0000 7fa8cc03c700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41a5d60 con 0x7fa8c4100c20 2024-02-05T07:51:03.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.640+0000 7fa8cc03c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41a5ed0 con 0x7fa8c41015c0 2024-02-05T07:51:03.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.640+0000 7fa8cc03c700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41a6090 con 0x7fa8c4106c60 2024-02-05T07:51:03.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.640+0000 7fa8c9dd8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8c4100c20 0x7fa8c4196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:51:03.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.640+0000 7fa8c9dd8700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8c4100c20 0x7fa8c4196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55048/0 (socket says 172.21.15.131:55048) 2024-02-05T07:51:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.640+0000 7fa8c9dd8700 1 -- 172.21.15.131:0/3452718113 learned_addr learned my addr 172.21.15.131:0/3452718113 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:51:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.640+0000 7fa8c95d7700 1 --2- 172.21.15.131:0/3452718113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8c41015c0 0x7fa8c4196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:51:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.641+0000 7fa8ca5d9700 1 --2- 172.21.15.131:0/3452718113 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa8c4106c60 0x7fa8c41970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:51:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.641+0000 7fa8c9dd8700 1 -- 172.21.15.131:0/3452718113 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa8c4106c60 msgr2=0x7fa8c41970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.641+0000 7fa8c9dd8700 1 --2- 172.21.15.131:0/3452718113 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa8c4106c60 0x7fa8c41970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.641+0000 7fa8c9dd8700 1 -- 172.21.15.131:0/3452718113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8c41015c0 msgr2=0x7fa8c4196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.641+0000 7fa8c9dd8700 1 --2- 172.21.15.131:0/3452718113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8c41015c0 0x7fa8c4196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.641+0000 7fa8c9dd8700 1 -- 172.21.15.131:0/3452718113 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b4009520 con 0x7fa8c4100c20 2024-02-05T07:51:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.641+0000 7fa8ca5d9700 1 --2- 172.21.15.131:0/3452718113 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa8c4106c60 0x7fa8c41970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T07:51:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.641+0000 7fa8c9dd8700 1 --2- 172.21.15.131:0/3452718113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8c4100c20 0x7fa8c4196620 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000c920 tx=0x7fa8c000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:51:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.641+0000 7fa8baffd700 1 -- 172.21.15.131:0/3452718113 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0007b00 con 0x7fa8c4100c20 2024-02-05T07:51:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.641+0000 7fa8cc03c700 1 -- 172.21.15.131:0/3452718113 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8bc009d00 con 0x7fa8c4100c20 2024-02-05T07:51:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.641+0000 7fa8cc03c700 1 -- 172.21.15.131:0/3452718113 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c41a6730 con 0x7fa8c4100c20 2024-02-05T07:51:03.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.641+0000 7fa8baffd700 1 -- 172.21.15.131:0/3452718113 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fa8c000ce70 con 0x7fa8c4100c20 2024-02-05T07:51:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.641+0000 7fa8baffd700 1 -- 172.21.15.131:0/3452718113 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0005470 con 0x7fa8c4100c20 2024-02-05T07:51:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.642+0000 7fa8cc03c700 1 -- 172.21.15.131:0/3452718113 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c4109180 con 0x7fa8c4100c20 2024-02-05T07:51:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.643+0000 7fa8baffd700 1 -- 172.21.15.131:0/3452718113 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fa8c000f450 con 0x7fa8c4100c20 2024-02-05T07:51:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.643+0000 7fa8baffd700 1 --2- 172.21.15.131:0/3452718113 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa8b006da80 0x7fa8b006ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:51:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.643+0000 7fa8baffd700 1 -- 172.21.15.131:0/3452718113 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fa8c008ce30 con 0x7fa8c4100c20 2024-02-05T07:51:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.643+0000 7fa8c95d7700 1 --2- 172.21.15.131:0/3452718113 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa8b006da80 0x7fa8b006ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:51:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.644+0000 7fa8c95d7700 1 --2- 172.21.15.131:0/3452718113 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa8b006da80 0x7fa8b006ff40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4198780 tx=0x7fa8b4009260 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:51:03.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.648+0000 7fa8baffd700 1 -- 172.21.15.131:0/3452718113 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa8c005a490 con 0x7fa8c4100c20 2024-02-05T07:51:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.840+0000 7fa8cc03c700 1 -- 172.21.15.131:0/3452718113 --> v2:172.21.15.131:6800/2614238408 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8c4069fc0 con 0x7fa8b006da80 2024-02-05T07:51:03.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.852+0000 7fa8baffd700 1 -- 172.21.15.131:0/3452718113 <== mgr.14571 v2:172.21.15.131:6800/2614238408 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+384405 (secure 0 0 0) 0x7fa8c4069fc0 con 0x7fa8b006da80 2024-02-05T07:51:03.857 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:51:03.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.855+0000 7fa8cc03c700 1 -- 172.21.15.131:0/3452718113 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa8b006da80 msgr2=0x7fa8b006ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.855+0000 7fa8cc03c700 1 --2- 172.21.15.131:0/3452718113 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa8b006da80 0x7fa8b006ff40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4198780 tx=0x7fa8b4009260 comp rx=0 tx=0).stop 2024-02-05T07:51:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.855+0000 7fa8cc03c700 1 -- 172.21.15.131:0/3452718113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8c4100c20 msgr2=0x7fa8c4196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.855+0000 7fa8cc03c700 1 --2- 172.21.15.131:0/3452718113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8c4100c20 0x7fa8c4196620 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000c920 tx=0x7fa8c000cc30 comp rx=0 tx=0).stop 2024-02-05T07:51:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.855+0000 7fa8cc03c700 1 -- 172.21.15.131:0/3452718113 shutdown_connections 2024-02-05T07:51:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.855+0000 7fa8cc03c700 1 --2- 172.21.15.131:0/3452718113 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa8b006da80 0x7fa8b006ff40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.855+0000 7fa8cc03c700 1 --2- 172.21.15.131:0/3452718113 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8c4100c20 0x7fa8c4196620 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.855+0000 7fa8cc03c700 1 --2- 172.21.15.131:0/3452718113 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa8c4106c60 0x7fa8c41970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.855+0000 7fa8cc03c700 1 --2- 172.21.15.131:0/3452718113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8c41015c0 0x7fa8c4196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.855+0000 7fa8cc03c700 1 -- 172.21.15.131:0/3452718113 >> 172.21.15.131:0/3452718113 conn(0x7fa8c4077b50 msgr2=0x7fa8c40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:51:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.855+0000 7fa8cc03c700 1 -- 172.21.15.131:0/3452718113 shutdown_connections 2024-02-05T07:51:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:03.855+0000 7fa8cc03c700 1 -- 172.21.15.131:0/3452718113 wait complete. 2024-02-05T07:51:03.861 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-02-05T07:51:04.985 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-02-05T07:51:03.271037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596316,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1003,"num_read_kb":849,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1771214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":240308,"kb_used_data":7516,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":749720908,"statfs":{"total":767960285184,"available":767714209792,"internally_reserved":0,"allocated":7696384,"data_stored":4078876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002715"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439220+0000","last_change":"2024-02-05T07:49:53.478936+0000","last_active":"2024-02-05T07:50:27.439220+0000","last_peered":"2024-02-05T07:50:27.439220+0000","last_clean":"2024-02-05T07:50:27.439220+0000","last_became_active":"2024-02-05T07:49:53.478668+0000","last_became_peered":"2024-02-05T07:49:53.478668+0000","last_unstale":"2024-02-05T07:50:27.439220+0000","last_undegraded":"2024-02-05T07:50:27.439220+0000","last_fullsized":"2024-02-05T07:50:27.439220+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:36:17.460657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333104+0000","last_change":"2024-02-05T07:49:51.468784+0000","last_active":"2024-02-05T07:50:27.333104+0000","last_peered":"2024-02-05T07:50:27.333104+0000","last_clean":"2024-02-05T07:50:27.333104+0000","last_became_active":"2024-02-05T07:49:51.468458+0000","last_became_peered":"2024-02-05T07:49:51.468458+0000","last_unstale":"2024-02-05T07:50:27.333104+0000","last_undegraded":"2024-02-05T07:50:27.333104+0000","last_fullsized":"2024-02-05T07:50:27.333104+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:52:57.305576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256230+0000","last_change":"2024-02-05T07:49:47.519575+0000","last_active":"2024-02-05T07:50:27.256230+0000","last_peered":"2024-02-05T07:50:27.256230+0000","last_clean":"2024-02-05T07:50:27.256230+0000","last_became_active":"2024-02-05T07:49:47.519170+0000","last_became_peered":"2024-02-05T07:49:47.519170+0000","last_unstale":"2024-02-05T07:50:27.256230+0000","last_undegraded":"2024-02-05T07:50:27.256230+0000","last_fullsized":"2024-02-05T07:50:27.256230+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:22:01.546829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439146+0000","last_change":"2024-02-05T07:49:49.447163+0000","last_active":"2024-02-05T07:50:27.439146+0000","last_peered":"2024-02-05T07:50:27.439146+0000","last_clean":"2024-02-05T07:50:27.439146+0000","last_became_active":"2024-02-05T07:49:49.446839+0000","last_became_peered":"2024-02-05T07:49:49.446839+0000","last_unstale":"2024-02-05T07:50:27.439146+0000","last_undegraded":"2024-02-05T07:50:27.439146+0000","last_fullsized":"2024-02-05T07:50:27.439146+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:00:53.571872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253816+0000","last_change":"2024-02-05T07:49:53.835672+0000","last_active":"2024-02-05T07:50:27.253816+0000","last_peered":"2024-02-05T07:50:27.253816+0000","last_clean":"2024-02-05T07:50:27.253816+0000","last_became_active":"2024-02-05T07:49:53.835335+0000","last_became_peered":"2024-02-05T07:49:53.835335+0000","last_unstale":"2024-02-05T07:50:27.253816+0000","last_undegraded":"2024-02-05T07:50:27.253816+0000","last_fullsized":"2024-02-05T07:50:27.253816+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:40:15.864719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439145+0000","last_change":"2024-02-05T07:49:51.466888+0000","last_active":"2024-02-05T07:50:27.439145+0000","last_peered":"2024-02-05T07:50:27.439145+0000","last_clean":"2024-02-05T07:50:27.439145+0000","last_became_active":"2024-02-05T07:49:51.466570+0000","last_became_peered":"2024-02-05T07:49:51.466570+0000","last_unstale":"2024-02-05T07:50:27.439145+0000","last_undegraded":"2024-02-05T07:50:27.439145+0000","last_fullsized":"2024-02-05T07:50:27.439145+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:40:05.187075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254439+0000","last_change":"2024-02-05T07:49:47.519371+0000","last_active":"2024-02-05T07:50:27.254439+0000","last_peered":"2024-02-05T07:50:27.254439+0000","last_clean":"2024-02-05T07:50:27.254439+0000","last_became_active":"2024-02-05T07:49:47.519066+0000","last_became_peered":"2024-02-05T07:49:47.519066+0000","last_unstale":"2024-02-05T07:50:27.254439+0000","last_undegraded":"2024-02-05T07:50:27.254439+0000","last_fullsized":"2024-02-05T07:50:27.254439+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:55:02.883024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252762+0000","last_change":"2024-02-05T07:49:49.454673+0000","last_active":"2024-02-05T07:50:27.252762+0000","last_peered":"2024-02-05T07:50:27.252762+0000","last_clean":"2024-02-05T07:50:27.252762+0000","last_became_active":"2024-02-05T07:49:49.454396+0000","last_became_peered":"2024-02-05T07:49:49.454396+0000","last_unstale":"2024-02-05T07:50:27.252762+0000","last_undegraded":"2024-02-05T07:50:27.252762+0000","last_fullsized":"2024-02-05T07:50:27.252762+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:34:38.007722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254592+0000","last_change":"2024-02-05T07:49:53.475148+0000","last_active":"2024-02-05T07:50:27.254592+0000","last_peered":"2024-02-05T07:50:27.254592+0000","last_clean":"2024-02-05T07:50:27.254592+0000","last_became_active":"2024-02-05T07:49:53.474422+0000","last_became_peered":"2024-02-05T07:49:53.474422+0000","last_unstale":"2024-02-05T07:50:27.254592+0000","last_undegraded":"2024-02-05T07:50:27.254592+0000","last_fullsized":"2024-02-05T07:50:27.254592+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:25:25.277540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359664+0000","last_change":"2024-02-05T07:49:51.465588+0000","last_active":"2024-02-05T07:50:27.359664+0000","last_peered":"2024-02-05T07:50:27.359664+0000","last_clean":"2024-02-05T07:50:27.359664+0000","last_became_active":"2024-02-05T07:49:51.465313+0000","last_became_peered":"2024-02-05T07:49:51.465313+0000","last_unstale":"2024-02-05T07:50:27.359664+0000","last_undegraded":"2024-02-05T07:50:27.359664+0000","last_fullsized":"2024-02-05T07:50:27.359664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:21:44.285873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359953+0000","last_change":"2024-02-05T07:49:47.451422+0000","last_active":"2024-02-05T07:50:27.359953+0000","last_peered":"2024-02-05T07:50:27.359953+0000","last_clean":"2024-02-05T07:50:27.359953+0000","last_became_active":"2024-02-05T07:49:47.450995+0000","last_became_peered":"2024-02-05T07:49:47.450995+0000","last_unstale":"2024-02-05T07:50:27.359953+0000","last_undegraded":"2024-02-05T07:50:27.359953+0000","last_fullsized":"2024-02-05T07:50:27.359953+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:53:01.222021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439385+0000","last_change":"2024-02-05T07:49:49.452032+0000","last_active":"2024-02-05T07:50:27.439385+0000","last_peered":"2024-02-05T07:50:27.439385+0000","last_clean":"2024-02-05T07:50:27.439385+0000","last_became_active":"2024-02-05T07:49:49.451230+0000","last_became_peered":"2024-02-05T07:49:49.451230+0000","last_unstale":"2024-02-05T07:50:27.439385+0000","last_undegraded":"2024-02-05T07:50:27.439385+0000","last_fullsized":"2024-02-05T07:50:27.439385+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:23:51.569743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359371+0000","last_change":"2024-02-05T07:49:53.481868+0000","last_active":"2024-02-05T07:50:27.359371+0000","last_peered":"2024-02-05T07:50:27.359371+0000","last_clean":"2024-02-05T07:50:27.359371+0000","last_became_active":"2024-02-05T07:49:53.481591+0000","last_became_peered":"2024-02-05T07:49:53.481591+0000","last_unstale":"2024-02-05T07:50:27.359371+0000","last_undegraded":"2024-02-05T07:50:27.359371+0000","last_fullsized":"2024-02-05T07:50:27.359371+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:48:20.838716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368394+0000","last_change":"2024-02-05T07:49:51.467253+0000","last_active":"2024-02-05T07:50:27.368394+0000","last_peered":"2024-02-05T07:50:27.368394+0000","last_clean":"2024-02-05T07:50:27.368394+0000","last_became_active":"2024-02-05T07:49:51.466963+0000","last_became_peered":"2024-02-05T07:49:51.466963+0000","last_unstale":"2024-02-05T07:50:27.368394+0000","last_undegraded":"2024-02-05T07:50:27.368394+0000","last_fullsized":"2024-02-05T07:50:27.368394+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:09:56.514606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359832+0000","last_change":"2024-02-05T07:49:47.447964+0000","last_active":"2024-02-05T07:50:27.359832+0000","last_peered":"2024-02-05T07:50:27.359832+0000","last_clean":"2024-02-05T07:50:27.359832+0000","last_became_active":"2024-02-05T07:49:47.447666+0000","last_became_peered":"2024-02-05T07:49:47.447666+0000","last_unstale":"2024-02-05T07:50:27.359832+0000","last_undegraded":"2024-02-05T07:50:27.359832+0000","last_fullsized":"2024-02-05T07:50:27.359832+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:27:28.586296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"70'5","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439543+0000","last_change":"2024-02-05T07:49:49.451900+0000","last_active":"2024-02-05T07:50:27.439543+0000","last_peered":"2024-02-05T07:50:27.439543+0000","last_clean":"2024-02-05T07:50:27.439543+0000","last_became_active":"2024-02-05T07:49:49.451587+0000","last_became_peered":"2024-02-05T07:49:49.451587+0000","last_unstale":"2024-02-05T07:50:27.439543+0000","last_undegraded":"2024-02-05T07:50:27.439543+0000","last_fullsized":"2024-02-05T07:50:27.439543+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:10:57.020252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.440239+0000","last_change":"2024-02-05T07:49:53.837827+0000","last_active":"2024-02-05T07:50:27.440239+0000","last_peered":"2024-02-05T07:50:27.440239+0000","last_clean":"2024-02-05T07:50:27.440239+0000","last_became_active":"2024-02-05T07:49:53.836685+0000","last_became_peered":"2024-02-05T07:49:53.836685+0000","last_unstale":"2024-02-05T07:50:27.440239+0000","last_undegraded":"2024-02-05T07:50:27.440239+0000","last_fullsized":"2024-02-05T07:50:27.440239+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:04:43.878414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333553+0000","last_change":"2024-02-05T07:49:51.466739+0000","last_active":"2024-02-05T07:50:27.333553+0000","last_peered":"2024-02-05T07:50:27.333553+0000","last_clean":"2024-02-05T07:50:27.333553+0000","last_became_active":"2024-02-05T07:49:51.466415+0000","last_became_peered":"2024-02-05T07:49:51.466415+0000","last_unstale":"2024-02-05T07:50:27.333553+0000","last_undegraded":"2024-02-05T07:50:27.333553+0000","last_fullsized":"2024-02-05T07:50:27.333553+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:01:51.310604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256608+0000","last_change":"2024-02-05T07:49:47.445972+0000","last_active":"2024-02-05T07:50:27.256608+0000","last_peered":"2024-02-05T07:50:27.256608+0000","last_clean":"2024-02-05T07:50:27.256608+0000","last_became_active":"2024-02-05T07:49:47.445408+0000","last_became_peered":"2024-02-05T07:49:47.445408+0000","last_unstale":"2024-02-05T07:50:27.256608+0000","last_undegraded":"2024-02-05T07:50:27.256608+0000","last_fullsized":"2024-02-05T07:50:27.256608+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:31:53.687093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.364945+0000","last_change":"2024-02-05T07:49:49.452582+0000","last_active":"2024-02-05T07:50:27.364945+0000","last_peered":"2024-02-05T07:50:27.364945+0000","last_clean":"2024-02-05T07:50:27.364945+0000","last_became_active":"2024-02-05T07:49:49.452268+0000","last_became_peered":"2024-02-05T07:49:49.452268+0000","last_unstale":"2024-02-05T07:50:27.364945+0000","last_undegraded":"2024-02-05T07:50:27.364945+0000","last_fullsized":"2024-02-05T07:50:27.364945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:23:50.259812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254136+0000","last_change":"2024-02-05T07:49:53.836986+0000","last_active":"2024-02-05T07:50:27.254136+0000","last_peered":"2024-02-05T07:50:27.254136+0000","last_clean":"2024-02-05T07:50:27.254136+0000","last_became_active":"2024-02-05T07:49:53.836599+0000","last_became_peered":"2024-02-05T07:49:53.836599+0000","last_unstale":"2024-02-05T07:50:27.254136+0000","last_undegraded":"2024-02-05T07:50:27.254136+0000","last_fullsized":"2024-02-05T07:50:27.254136+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:08:33.779724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439855+0000","last_change":"2024-02-05T07:49:51.469272+0000","last_active":"2024-02-05T07:50:27.439855+0000","last_peered":"2024-02-05T07:50:27.439855+0000","last_clean":"2024-02-05T07:50:27.439855+0000","last_became_active":"2024-02-05T07:49:51.469006+0000","last_became_peered":"2024-02-05T07:49:51.469006+0000","last_unstale":"2024-02-05T07:50:27.439855+0000","last_undegraded":"2024-02-05T07:50:27.439855+0000","last_fullsized":"2024-02-05T07:50:27.439855+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:40:56.285764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439463+0000","last_change":"2024-02-05T07:49:47.520229+0000","last_active":"2024-02-05T07:50:27.439463+0000","last_peered":"2024-02-05T07:50:27.439463+0000","last_clean":"2024-02-05T07:50:27.439463+0000","last_became_active":"2024-02-05T07:49:47.519898+0000","last_became_peered":"2024-02-05T07:49:47.519898+0000","last_unstale":"2024-02-05T07:50:27.439463+0000","last_undegraded":"2024-02-05T07:50:27.439463+0000","last_fullsized":"2024-02-05T07:50:27.439463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:42:11.731508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"70'12","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254756+0000","last_change":"2024-02-05T07:49:49.454016+0000","last_active":"2024-02-05T07:50:27.254756+0000","last_peered":"2024-02-05T07:50:27.254756+0000","last_clean":"2024-02-05T07:50:27.254756+0000","last_became_active":"2024-02-05T07:49:49.453713+0000","last_became_peered":"2024-02-05T07:49:49.453713+0000","last_unstale":"2024-02-05T07:50:27.254756+0000","last_undegraded":"2024-02-05T07:50:27.254756+0000","last_fullsized":"2024-02-05T07:50:27.254756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:11:42.647053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"70'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368746+0000","last_change":"2024-02-05T07:49:53.480710+0000","last_active":"2024-02-05T07:50:27.368746+0000","last_peered":"2024-02-05T07:50:27.368746+0000","last_clean":"2024-02-05T07:50:27.368746+0000","last_became_active":"2024-02-05T07:49:53.480449+0000","last_became_peered":"2024-02-05T07:49:53.480449+0000","last_unstale":"2024-02-05T07:50:27.368746+0000","last_undegraded":"2024-02-05T07:50:27.368746+0000","last_fullsized":"2024-02-05T07:50:27.368746+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:19:24.609571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"70'8","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:57.574507+0000","last_change":"2024-02-05T07:49:51.463317+0000","last_active":"2024-02-05T07:50:57.574507+0000","last_peered":"2024-02-05T07:50:57.574507+0000","last_clean":"2024-02-05T07:50:57.574507+0000","last_became_active":"2024-02-05T07:49:51.463042+0000","last_became_peered":"2024-02-05T07:49:51.463042+0000","last_unstale":"2024-02-05T07:50:57.574507+0000","last_undegraded":"2024-02-05T07:50:57.574507+0000","last_fullsized":"2024-02-05T07:50:57.574507+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:54:26.640021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"63'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333165+0000","last_change":"2024-02-05T07:49:47.447165+0000","last_active":"2024-02-05T07:50:27.333165+0000","last_peered":"2024-02-05T07:50:27.333165+0000","last_clean":"2024-02-05T07:50:27.333165+0000","last_became_active":"2024-02-05T07:49:47.446862+0000","last_became_peered":"2024-02-05T07:49:47.446862+0000","last_unstale":"2024-02-05T07:50:27.333165+0000","last_undegraded":"2024-02-05T07:50:27.333165+0000","last_fullsized":"2024-02-05T07:50:27.333165+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:46:43.539023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256606+0000","last_change":"2024-02-05T07:49:49.735032+0000","last_active":"2024-02-05T07:50:27.256606+0000","last_peered":"2024-02-05T07:50:27.256606+0000","last_clean":"2024-02-05T07:50:27.256606+0000","last_became_active":"2024-02-05T07:49:49.734750+0000","last_became_peered":"2024-02-05T07:49:49.734750+0000","last_unstale":"2024-02-05T07:50:27.256606+0000","last_undegraded":"2024-02-05T07:50:27.256606+0000","last_fullsized":"2024-02-05T07:50:27.256606+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:53:41.822618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333354+0000","last_change":"2024-02-05T07:49:53.474114+0000","last_active":"2024-02-05T07:50:27.333354+0000","last_peered":"2024-02-05T07:50:27.333354+0000","last_clean":"2024-02-05T07:50:27.333354+0000","last_became_active":"2024-02-05T07:49:53.473808+0000","last_became_peered":"2024-02-05T07:49:53.473808+0000","last_unstale":"2024-02-05T07:50:27.333354+0000","last_undegraded":"2024-02-05T07:50:27.333354+0000","last_fullsized":"2024-02-05T07:50:27.333354+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:39:52.085855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256847+0000","last_change":"2024-02-05T07:49:51.462445+0000","last_active":"2024-02-05T07:50:27.256847+0000","last_peered":"2024-02-05T07:50:27.256847+0000","last_clean":"2024-02-05T07:50:27.256847+0000","last_became_active":"2024-02-05T07:49:51.461679+0000","last_became_peered":"2024-02-05T07:49:51.461679+0000","last_unstale":"2024-02-05T07:50:27.256847+0000","last_undegraded":"2024-02-05T07:50:27.256847+0000","last_fullsized":"2024-02-05T07:50:27.256847+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:40:46.347573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252165+0000","last_change":"2024-02-05T07:49:47.443560+0000","last_active":"2024-02-05T07:50:27.252165+0000","last_peered":"2024-02-05T07:50:27.252165+0000","last_clean":"2024-02-05T07:50:27.252165+0000","last_became_active":"2024-02-05T07:49:47.443309+0000","last_became_peered":"2024-02-05T07:49:47.443309+0000","last_unstale":"2024-02-05T07:50:27.252165+0000","last_undegraded":"2024-02-05T07:50:27.252165+0000","last_fullsized":"2024-02-05T07:50:27.252165+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:26:54.714704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254073+0000","last_change":"2024-02-05T07:49:49.446484+0000","last_active":"2024-02-05T07:50:27.254073+0000","last_peered":"2024-02-05T07:50:27.254073+0000","last_clean":"2024-02-05T07:50:27.254073+0000","last_became_active":"2024-02-05T07:49:49.446165+0000","last_became_peered":"2024-02-05T07:49:49.446165+0000","last_unstale":"2024-02-05T07:50:27.254073+0000","last_undegraded":"2024-02-05T07:50:27.254073+0000","last_fullsized":"2024-02-05T07:50:27.254073+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:38:51.811836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368391+0000","last_change":"2024-02-05T07:49:53.480539+0000","last_active":"2024-02-05T07:50:27.368391+0000","last_peered":"2024-02-05T07:50:27.368391+0000","last_clean":"2024-02-05T07:50:27.368391+0000","last_became_active":"2024-02-05T07:49:53.480241+0000","last_became_peered":"2024-02-05T07:49:53.480241+0000","last_unstale":"2024-02-05T07:50:27.368391+0000","last_undegraded":"2024-02-05T07:50:27.368391+0000","last_fullsized":"2024-02-05T07:50:27.368391+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:40:28.872638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253088+0000","last_change":"2024-02-05T07:49:51.460300+0000","last_active":"2024-02-05T07:50:27.253088+0000","last_peered":"2024-02-05T07:50:27.253088+0000","last_clean":"2024-02-05T07:50:27.253088+0000","last_became_active":"2024-02-05T07:49:51.460032+0000","last_became_peered":"2024-02-05T07:49:51.460032+0000","last_unstale":"2024-02-05T07:50:27.253088+0000","last_undegraded":"2024-02-05T07:50:27.253088+0000","last_fullsized":"2024-02-05T07:50:27.253088+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:31:52.284614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"63'1","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256122+0000","last_change":"2024-02-05T07:49:47.447541+0000","last_active":"2024-02-05T07:50:27.256122+0000","last_peered":"2024-02-05T07:50:27.256122+0000","last_clean":"2024-02-05T07:50:27.256122+0000","last_became_active":"2024-02-05T07:49:47.447220+0000","last_became_peered":"2024-02-05T07:49:47.447220+0000","last_unstale":"2024-02-05T07:50:27.256122+0000","last_undegraded":"2024-02-05T07:50:27.256122+0000","last_fullsized":"2024-02-05T07:50:27.256122+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:19:56.908993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"70'4","reported_seq":35,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252652+0000","last_change":"2024-02-05T07:49:49.455803+0000","last_active":"2024-02-05T07:50:27.252652+0000","last_peered":"2024-02-05T07:50:27.252652+0000","last_clean":"2024-02-05T07:50:27.252652+0000","last_became_active":"2024-02-05T07:49:49.455410+0000","last_became_peered":"2024-02-05T07:49:49.455410+0000","last_unstale":"2024-02-05T07:50:27.252652+0000","last_undegraded":"2024-02-05T07:50:27.252652+0000","last_fullsized":"2024-02-05T07:50:27.252652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:24:44.281857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253655+0000","last_change":"2024-02-05T07:49:53.835566+0000","last_active":"2024-02-05T07:50:27.253655+0000","last_peered":"2024-02-05T07:50:27.253655+0000","last_clean":"2024-02-05T07:50:27.253655+0000","last_became_active":"2024-02-05T07:49:53.835262+0000","last_became_peered":"2024-02-05T07:49:53.835262+0000","last_unstale":"2024-02-05T07:50:27.253655+0000","last_undegraded":"2024-02-05T07:50:27.253655+0000","last_fullsized":"2024-02-05T07:50:27.253655+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:09:54.372653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368695+0000","last_change":"2024-02-05T07:49:51.464304+0000","last_active":"2024-02-05T07:50:27.368695+0000","last_peered":"2024-02-05T07:50:27.368695+0000","last_clean":"2024-02-05T07:50:27.368695+0000","last_became_active":"2024-02-05T07:49:51.464012+0000","last_became_peered":"2024-02-05T07:49:51.464012+0000","last_unstale":"2024-02-05T07:50:27.368695+0000","last_undegraded":"2024-02-05T07:50:27.368695+0000","last_fullsized":"2024-02-05T07:50:27.368695+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:37:36.201341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.360033+0000","last_change":"2024-02-05T07:49:47.446196+0000","last_active":"2024-02-05T07:50:27.360033+0000","last_peered":"2024-02-05T07:50:27.360033+0000","last_clean":"2024-02-05T07:50:27.360033+0000","last_became_active":"2024-02-05T07:49:47.445869+0000","last_became_peered":"2024-02-05T07:49:47.445869+0000","last_unstale":"2024-02-05T07:50:27.360033+0000","last_undegraded":"2024-02-05T07:50:27.360033+0000","last_fullsized":"2024-02-05T07:50:27.360033+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:40:35.985397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252651+0000","last_change":"2024-02-05T07:49:49.734819+0000","last_active":"2024-02-05T07:50:27.252651+0000","last_peered":"2024-02-05T07:50:27.252651+0000","last_clean":"2024-02-05T07:50:27.252651+0000","last_became_active":"2024-02-05T07:49:49.734577+0000","last_became_peered":"2024-02-05T07:49:49.734577+0000","last_unstale":"2024-02-05T07:50:27.252651+0000","last_undegraded":"2024-02-05T07:50:27.252651+0000","last_fullsized":"2024-02-05T07:50:27.252651+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:33:18.467860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254283+0000","last_change":"2024-02-05T07:49:53.475769+0000","last_active":"2024-02-05T07:50:27.254283+0000","last_peered":"2024-02-05T07:50:27.254283+0000","last_clean":"2024-02-05T07:50:27.254283+0000","last_became_active":"2024-02-05T07:49:53.475473+0000","last_became_peered":"2024-02-05T07:49:53.475473+0000","last_unstale":"2024-02-05T07:50:27.254283+0000","last_undegraded":"2024-02-05T07:50:27.254283+0000","last_fullsized":"2024-02-05T07:50:27.254283+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:14:53.728317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253836+0000","last_change":"2024-02-05T07:49:51.466176+0000","last_active":"2024-02-05T07:50:27.253836+0000","last_peered":"2024-02-05T07:50:27.253836+0000","last_clean":"2024-02-05T07:50:27.253836+0000","last_became_active":"2024-02-05T07:49:51.464949+0000","last_became_peered":"2024-02-05T07:49:51.464949+0000","last_unstale":"2024-02-05T07:50:27.253836+0000","last_undegraded":"2024-02-05T07:50:27.253836+0000","last_fullsized":"2024-02-05T07:50:27.253836+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:22:05.181262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368862+0000","last_change":"2024-02-05T07:49:47.450946+0000","last_active":"2024-02-05T07:50:27.368862+0000","last_peered":"2024-02-05T07:50:27.368862+0000","last_clean":"2024-02-05T07:50:27.368862+0000","last_became_active":"2024-02-05T07:49:47.450675+0000","last_became_peered":"2024-02-05T07:49:47.450675+0000","last_unstale":"2024-02-05T07:50:27.368862+0000","last_undegraded":"2024-02-05T07:50:27.368862+0000","last_fullsized":"2024-02-05T07:50:27.368862+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:51:55.185524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333769+0000","last_change":"2024-02-05T07:49:49.452987+0000","last_active":"2024-02-05T07:50:27.333769+0000","last_peered":"2024-02-05T07:50:27.333769+0000","last_clean":"2024-02-05T07:50:27.333769+0000","last_became_active":"2024-02-05T07:49:49.452710+0000","last_became_peered":"2024-02-05T07:49:49.452710+0000","last_unstale":"2024-02-05T07:50:27.333769+0000","last_undegraded":"2024-02-05T07:50:27.333769+0000","last_fullsized":"2024-02-05T07:50:27.333769+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:30:34.871151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252277+0000","last_change":"2024-02-05T07:49:53.481105+0000","last_active":"2024-02-05T07:50:27.252277+0000","last_peered":"2024-02-05T07:50:27.252277+0000","last_clean":"2024-02-05T07:50:27.252277+0000","last_became_active":"2024-02-05T07:49:53.480568+0000","last_became_peered":"2024-02-05T07:49:53.480568+0000","last_unstale":"2024-02-05T07:50:27.252277+0000","last_undegraded":"2024-02-05T07:50:27.252277+0000","last_fullsized":"2024-02-05T07:50:27.252277+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:18:36.474182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333668+0000","last_change":"2024-02-05T07:49:51.469593+0000","last_active":"2024-02-05T07:50:27.333668+0000","last_peered":"2024-02-05T07:50:27.333668+0000","last_clean":"2024-02-05T07:50:27.333668+0000","last_became_active":"2024-02-05T07:49:51.469274+0000","last_became_peered":"2024-02-05T07:49:51.469274+0000","last_unstale":"2024-02-05T07:50:27.333668+0000","last_undegraded":"2024-02-05T07:50:27.333668+0000","last_fullsized":"2024-02-05T07:50:27.333668+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:58:09.801021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.440045+0000","last_change":"2024-02-05T07:49:47.453529+0000","last_active":"2024-02-05T07:50:27.440045+0000","last_peered":"2024-02-05T07:50:27.440045+0000","last_clean":"2024-02-05T07:50:27.440045+0000","last_became_active":"2024-02-05T07:49:47.453167+0000","last_became_peered":"2024-02-05T07:49:47.453167+0000","last_unstale":"2024-02-05T07:50:27.440045+0000","last_undegraded":"2024-02-05T07:50:27.440045+0000","last_fullsized":"2024-02-05T07:50:27.440045+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:18:30.336500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439702+0000","last_change":"2024-02-05T07:49:49.445760+0000","last_active":"2024-02-05T07:50:27.439702+0000","last_peered":"2024-02-05T07:50:27.439702+0000","last_clean":"2024-02-05T07:50:27.439702+0000","last_became_active":"2024-02-05T07:49:49.445287+0000","last_became_peered":"2024-02-05T07:49:49.445287+0000","last_unstale":"2024-02-05T07:50:27.439702+0000","last_undegraded":"2024-02-05T07:50:27.439702+0000","last_fullsized":"2024-02-05T07:50:27.439702+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:33:06.759850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256289+0000","last_change":"2024-02-05T07:49:53.481538+0000","last_active":"2024-02-05T07:50:27.256289+0000","last_peered":"2024-02-05T07:50:27.256289+0000","last_clean":"2024-02-05T07:50:27.256289+0000","last_became_active":"2024-02-05T07:49:53.481176+0000","last_became_peered":"2024-02-05T07:49:53.481176+0000","last_unstale":"2024-02-05T07:50:27.256289+0000","last_undegraded":"2024-02-05T07:50:27.256289+0000","last_fullsized":"2024-02-05T07:50:27.256289+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:16:32.895287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:57.574486+0000","last_change":"2024-02-05T07:49:51.469592+0000","last_active":"2024-02-05T07:50:57.574486+0000","last_peered":"2024-02-05T07:50:57.574486+0000","last_clean":"2024-02-05T07:50:57.574486+0000","last_became_active":"2024-02-05T07:49:51.469322+0000","last_became_peered":"2024-02-05T07:49:51.469322+0000","last_unstale":"2024-02-05T07:50:57.574486+0000","last_undegraded":"2024-02-05T07:50:57.574486+0000","last_fullsized":"2024-02-05T07:50:57.574486+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:49:35.499256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368461+0000","last_change":"2024-02-05T07:49:47.519907+0000","last_active":"2024-02-05T07:50:27.368461+0000","last_peered":"2024-02-05T07:50:27.368461+0000","last_clean":"2024-02-05T07:50:27.368461+0000","last_became_active":"2024-02-05T07:49:47.519608+0000","last_became_peered":"2024-02-05T07:49:47.519608+0000","last_unstale":"2024-02-05T07:50:27.368461+0000","last_undegraded":"2024-02-05T07:50:27.368461+0000","last_fullsized":"2024-02-05T07:50:27.368461+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:34:11.853473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"70'10","reported_seq":62,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252610+0000","last_change":"2024-02-05T07:50:11.872571+0000","last_active":"2024-02-05T07:50:27.252610+0000","last_peered":"2024-02-05T07:50:27.252610+0000","last_clean":"2024-02-05T07:50:27.252610+0000","last_became_active":"2024-02-05T07:50:11.861148+0000","last_became_peered":"2024-02-05T07:50:11.861148+0000","last_unstale":"2024-02-05T07:50:27.252610+0000","last_undegraded":"2024-02-05T07:50:27.252610+0000","last_fullsized":"2024-02-05T07:50:27.252610+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:42:58.286059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.440364+0000","last_change":"2024-02-05T07:49:53.481966+0000","last_active":"2024-02-05T07:50:27.440364+0000","last_peered":"2024-02-05T07:50:27.440364+0000","last_clean":"2024-02-05T07:50:27.440364+0000","last_became_active":"2024-02-05T07:49:53.481119+0000","last_became_peered":"2024-02-05T07:49:53.481119+0000","last_unstale":"2024-02-05T07:50:27.440364+0000","last_undegraded":"2024-02-05T07:50:27.440364+0000","last_fullsized":"2024-02-05T07:50:27.440364+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:26:52.284632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:57.574356+0000","last_change":"2024-02-05T07:49:51.469279+0000","last_active":"2024-02-05T07:50:57.574356+0000","last_peered":"2024-02-05T07:50:57.574356+0000","last_clean":"2024-02-05T07:50:57.574356+0000","last_became_active":"2024-02-05T07:49:51.468985+0000","last_became_peered":"2024-02-05T07:49:51.468985+0000","last_unstale":"2024-02-05T07:50:57.574356+0000","last_undegraded":"2024-02-05T07:50:57.574356+0000","last_fullsized":"2024-02-05T07:50:57.574356+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:23:51.463404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.255389+0000","last_change":"2024-02-05T07:49:47.438685+0000","last_active":"2024-02-05T07:50:27.255389+0000","last_peered":"2024-02-05T07:50:27.255389+0000","last_clean":"2024-02-05T07:50:27.255389+0000","last_became_active":"2024-02-05T07:49:47.438378+0000","last_became_peered":"2024-02-05T07:49:47.438378+0000","last_unstale":"2024-02-05T07:50:27.255389+0000","last_undegraded":"2024-02-05T07:50:27.255389+0000","last_fullsized":"2024-02-05T07:50:27.255389+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:07:59.061901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.360173+0000","last_change":"2024-02-05T07:49:49.735895+0000","last_active":"2024-02-05T07:50:27.360173+0000","last_peered":"2024-02-05T07:50:27.360173+0000","last_clean":"2024-02-05T07:50:27.360173+0000","last_became_active":"2024-02-05T07:49:49.735596+0000","last_became_peered":"2024-02-05T07:49:49.735596+0000","last_unstale":"2024-02-05T07:50:27.360173+0000","last_undegraded":"2024-02-05T07:50:27.360173+0000","last_fullsized":"2024-02-05T07:50:27.360173+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:40:42.036089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.365135+0000","last_change":"2024-02-05T07:49:53.485285+0000","last_active":"2024-02-05T07:50:27.365135+0000","last_peered":"2024-02-05T07:50:27.365135+0000","last_clean":"2024-02-05T07:50:27.365135+0000","last_became_active":"2024-02-05T07:49:53.482379+0000","last_became_peered":"2024-02-05T07:49:53.482379+0000","last_unstale":"2024-02-05T07:50:27.365135+0000","last_undegraded":"2024-02-05T07:50:27.365135+0000","last_fullsized":"2024-02-05T07:50:27.365135+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:45:51.848267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254565+0000","last_change":"2024-02-05T07:49:51.472049+0000","last_active":"2024-02-05T07:50:27.254565+0000","last_peered":"2024-02-05T07:50:27.254565+0000","last_clean":"2024-02-05T07:50:27.254565+0000","last_became_active":"2024-02-05T07:49:51.471747+0000","last_became_peered":"2024-02-05T07:49:51.471747+0000","last_unstale":"2024-02-05T07:50:27.254565+0000","last_undegraded":"2024-02-05T07:50:27.254565+0000","last_fullsized":"2024-02-05T07:50:27.254565+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:51:35.136254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368392+0000","last_change":"2024-02-05T07:49:47.450415+0000","last_active":"2024-02-05T07:50:27.368392+0000","last_peered":"2024-02-05T07:50:27.368392+0000","last_clean":"2024-02-05T07:50:27.368392+0000","last_became_active":"2024-02-05T07:49:47.448698+0000","last_became_peered":"2024-02-05T07:49:47.448698+0000","last_unstale":"2024-02-05T07:50:27.368392+0000","last_undegraded":"2024-02-05T07:50:27.368392+0000","last_fullsized":"2024-02-05T07:50:27.368392+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:09:40.857335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256728+0000","last_change":"2024-02-05T07:49:49.735988+0000","last_active":"2024-02-05T07:50:27.256728+0000","last_peered":"2024-02-05T07:50:27.256728+0000","last_clean":"2024-02-05T07:50:27.256728+0000","last_became_active":"2024-02-05T07:49:49.735650+0000","last_became_peered":"2024-02-05T07:49:49.735650+0000","last_unstale":"2024-02-05T07:50:27.256728+0000","last_undegraded":"2024-02-05T07:50:27.256728+0000","last_fullsized":"2024-02-05T07:50:27.256728+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:02:56.402985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368804+0000","last_change":"2024-02-05T07:49:53.836415+0000","last_active":"2024-02-05T07:50:27.368804+0000","last_peered":"2024-02-05T07:50:27.368804+0000","last_clean":"2024-02-05T07:50:27.368804+0000","last_became_active":"2024-02-05T07:49:53.836158+0000","last_became_peered":"2024-02-05T07:49:53.836158+0000","last_unstale":"2024-02-05T07:50:27.368804+0000","last_undegraded":"2024-02-05T07:50:27.368804+0000","last_fullsized":"2024-02-05T07:50:27.368804+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:46:25.206392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.360122+0000","last_change":"2024-02-05T07:49:51.468826+0000","last_active":"2024-02-05T07:50:27.360122+0000","last_peered":"2024-02-05T07:50:27.360122+0000","last_clean":"2024-02-05T07:50:27.360122+0000","last_became_active":"2024-02-05T07:49:51.468529+0000","last_became_peered":"2024-02-05T07:49:51.468529+0000","last_unstale":"2024-02-05T07:50:27.360122+0000","last_undegraded":"2024-02-05T07:50:27.360122+0000","last_fullsized":"2024-02-05T07:50:27.360122+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:10:49.229035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253031+0000","last_change":"2024-02-05T07:49:47.447847+0000","last_active":"2024-02-05T07:50:27.253031+0000","last_peered":"2024-02-05T07:50:27.253031+0000","last_clean":"2024-02-05T07:50:27.253031+0000","last_became_active":"2024-02-05T07:49:47.447557+0000","last_became_peered":"2024-02-05T07:49:47.447557+0000","last_unstale":"2024-02-05T07:50:27.253031+0000","last_undegraded":"2024-02-05T07:50:27.253031+0000","last_fullsized":"2024-02-05T07:50:27.253031+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:39:14.469684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"70'6","reported_seq":38,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254451+0000","last_change":"2024-02-05T07:49:49.735684+0000","last_active":"2024-02-05T07:50:27.254451+0000","last_peered":"2024-02-05T07:50:27.254451+0000","last_clean":"2024-02-05T07:50:27.254451+0000","last_became_active":"2024-02-05T07:49:49.735297+0000","last_became_peered":"2024-02-05T07:49:49.735297+0000","last_unstale":"2024-02-05T07:50:27.254451+0000","last_undegraded":"2024-02-05T07:50:27.254451+0000","last_fullsized":"2024-02-05T07:50:27.254451+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:55:48.643476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"70'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359843+0000","last_change":"2024-02-05T07:49:53.834941+0000","last_active":"2024-02-05T07:50:27.359843+0000","last_peered":"2024-02-05T07:50:27.359843+0000","last_clean":"2024-02-05T07:50:27.359843+0000","last_became_active":"2024-02-05T07:49:53.834635+0000","last_became_peered":"2024-02-05T07:49:53.834635+0000","last_unstale":"2024-02-05T07:50:27.359843+0000","last_undegraded":"2024-02-05T07:50:27.359843+0000","last_fullsized":"2024-02-05T07:50:27.359843+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:48:15.158738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"63'2","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.369115+0000","last_change":"2024-02-05T07:49:47.448888+0000","last_active":"2024-02-05T07:50:27.369115+0000","last_peered":"2024-02-05T07:50:27.369115+0000","last_clean":"2024-02-05T07:50:27.369115+0000","last_became_active":"2024-02-05T07:49:47.448550+0000","last_became_peered":"2024-02-05T07:49:47.448550+0000","last_unstale":"2024-02-05T07:50:27.369115+0000","last_undegraded":"2024-02-05T07:50:27.369115+0000","last_fullsized":"2024-02-05T07:50:27.369115+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:26:28.653523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359435+0000","last_change":"2024-02-05T07:49:49.737467+0000","last_active":"2024-02-05T07:50:27.359435+0000","last_peered":"2024-02-05T07:50:27.359435+0000","last_clean":"2024-02-05T07:50:27.359435+0000","last_became_active":"2024-02-05T07:49:49.737086+0000","last_became_peered":"2024-02-05T07:49:49.737086+0000","last_unstale":"2024-02-05T07:50:27.359435+0000","last_undegraded":"2024-02-05T07:50:27.359435+0000","last_fullsized":"2024-02-05T07:50:27.359435+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:31:52.911508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:57.574508+0000","last_change":"2024-02-05T07:49:51.464483+0000","last_active":"2024-02-05T07:50:57.574508+0000","last_peered":"2024-02-05T07:50:57.574508+0000","last_clean":"2024-02-05T07:50:57.574508+0000","last_became_active":"2024-02-05T07:49:51.464208+0000","last_became_peered":"2024-02-05T07:49:51.464208+0000","last_unstale":"2024-02-05T07:50:57.574508+0000","last_undegraded":"2024-02-05T07:50:57.574508+0000","last_fullsized":"2024-02-05T07:50:57.574508+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:41:49.063996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253747+0000","last_change":"2024-02-05T07:49:53.482133+0000","last_active":"2024-02-05T07:50:27.253747+0000","last_peered":"2024-02-05T07:50:27.253747+0000","last_clean":"2024-02-05T07:50:27.253747+0000","last_became_active":"2024-02-05T07:49:53.481520+0000","last_became_peered":"2024-02-05T07:49:53.481520+0000","last_unstale":"2024-02-05T07:50:27.253747+0000","last_undegraded":"2024-02-05T07:50:27.253747+0000","last_fullsized":"2024-02-05T07:50:27.253747+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:54:15.842757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.369197+0000","last_change":"2024-02-05T07:49:47.449999+0000","last_active":"2024-02-05T07:50:27.369197+0000","last_peered":"2024-02-05T07:50:27.369197+0000","last_clean":"2024-02-05T07:50:27.369197+0000","last_became_active":"2024-02-05T07:49:47.448351+0000","last_became_peered":"2024-02-05T07:49:47.448351+0000","last_unstale":"2024-02-05T07:50:27.369197+0000","last_undegraded":"2024-02-05T07:50:27.369197+0000","last_fullsized":"2024-02-05T07:50:27.369197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:03:49.578642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"70'12","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439438+0000","last_change":"2024-02-05T07:49:49.451505+0000","last_active":"2024-02-05T07:50:27.439438+0000","last_peered":"2024-02-05T07:50:27.439438+0000","last_clean":"2024-02-05T07:50:27.439438+0000","last_became_active":"2024-02-05T07:49:49.451159+0000","last_became_peered":"2024-02-05T07:49:49.451159+0000","last_unstale":"2024-02-05T07:50:27.439438+0000","last_undegraded":"2024-02-05T07:50:27.439438+0000","last_fullsized":"2024-02-05T07:50:27.439438+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:53:36.284342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.364494+0000","last_change":"2024-02-05T07:49:51.469984+0000","last_active":"2024-02-05T07:50:27.364494+0000","last_peered":"2024-02-05T07:50:27.364494+0000","last_clean":"2024-02-05T07:50:27.364494+0000","last_became_active":"2024-02-05T07:49:51.469660+0000","last_became_peered":"2024-02-05T07:49:51.469660+0000","last_unstale":"2024-02-05T07:50:27.364494+0000","last_undegraded":"2024-02-05T07:50:27.364494+0000","last_fullsized":"2024-02-05T07:50:27.364494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:17:57.565953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368197+0000","last_change":"2024-02-05T07:49:53.484776+0000","last_active":"2024-02-05T07:50:27.368197+0000","last_peered":"2024-02-05T07:50:27.368197+0000","last_clean":"2024-02-05T07:50:27.368197+0000","last_became_active":"2024-02-05T07:49:53.484428+0000","last_became_peered":"2024-02-05T07:49:53.484428+0000","last_unstale":"2024-02-05T07:50:27.368197+0000","last_undegraded":"2024-02-05T07:50:27.368197+0000","last_fullsized":"2024-02-05T07:50:27.368197+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:35:23.891153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.364829+0000","last_change":"2024-02-05T07:49:51.461096+0000","last_active":"2024-02-05T07:50:27.364829+0000","last_peered":"2024-02-05T07:50:27.364829+0000","last_clean":"2024-02-05T07:50:27.364829+0000","last_became_active":"2024-02-05T07:49:51.460803+0000","last_became_peered":"2024-02-05T07:49:51.460803+0000","last_unstale":"2024-02-05T07:50:27.364829+0000","last_undegraded":"2024-02-05T07:50:27.364829+0000","last_fullsized":"2024-02-05T07:50:27.364829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:09:55.228750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.369025+0000","last_change":"2024-02-05T07:49:47.451314+0000","last_active":"2024-02-05T07:50:27.369025+0000","last_peered":"2024-02-05T07:50:27.369025+0000","last_clean":"2024-02-05T07:50:27.369025+0000","last_became_active":"2024-02-05T07:49:47.450794+0000","last_became_peered":"2024-02-05T07:49:47.450794+0000","last_unstale":"2024-02-05T07:50:27.369025+0000","last_undegraded":"2024-02-05T07:50:27.369025+0000","last_fullsized":"2024-02-05T07:50:27.369025+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:32:59.395201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"70'19","reported_seq":60,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253374+0000","last_change":"2024-02-05T07:49:49.734559+0000","last_active":"2024-02-05T07:50:27.253374+0000","last_peered":"2024-02-05T07:50:27.253374+0000","last_clean":"2024-02-05T07:50:27.253374+0000","last_became_active":"2024-02-05T07:49:49.734204+0000","last_became_peered":"2024-02-05T07:49:49.734204+0000","last_unstale":"2024-02-05T07:50:27.253374+0000","last_undegraded":"2024-02-05T07:50:27.253374+0000","last_fullsized":"2024-02-05T07:50:27.253374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:47:19.559079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254920+0000","last_change":"2024-02-05T07:49:53.477215+0000","last_active":"2024-02-05T07:50:27.254920+0000","last_peered":"2024-02-05T07:50:27.254920+0000","last_clean":"2024-02-05T07:50:27.254920+0000","last_became_active":"2024-02-05T07:49:53.476921+0000","last_became_peered":"2024-02-05T07:49:53.476921+0000","last_unstale":"2024-02-05T07:50:27.254920+0000","last_undegraded":"2024-02-05T07:50:27.254920+0000","last_fullsized":"2024-02-05T07:50:27.254920+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:07:12.633466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.364782+0000","last_change":"2024-02-05T07:49:51.465329+0000","last_active":"2024-02-05T07:50:27.364782+0000","last_peered":"2024-02-05T07:50:27.364782+0000","last_clean":"2024-02-05T07:50:27.364782+0000","last_became_active":"2024-02-05T07:49:51.464968+0000","last_became_peered":"2024-02-05T07:49:51.464968+0000","last_unstale":"2024-02-05T07:50:27.364782+0000","last_undegraded":"2024-02-05T07:50:27.364782+0000","last_fullsized":"2024-02-05T07:50:27.364782+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:06:39.051173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359874+0000","last_change":"2024-02-05T07:49:47.450354+0000","last_active":"2024-02-05T07:50:27.359874+0000","last_peered":"2024-02-05T07:50:27.359874+0000","last_clean":"2024-02-05T07:50:27.359874+0000","last_became_active":"2024-02-05T07:49:47.450004+0000","last_became_peered":"2024-02-05T07:49:47.450004+0000","last_unstale":"2024-02-05T07:50:27.359874+0000","last_undegraded":"2024-02-05T07:50:27.359874+0000","last_fullsized":"2024-02-05T07:50:27.359874+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:06:36.695339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256489+0000","last_change":"2024-02-05T07:49:49.448567+0000","last_active":"2024-02-05T07:50:27.256489+0000","last_peered":"2024-02-05T07:50:27.256489+0000","last_clean":"2024-02-05T07:50:27.256489+0000","last_became_active":"2024-02-05T07:49:49.448168+0000","last_became_peered":"2024-02-05T07:49:49.448168+0000","last_unstale":"2024-02-05T07:50:27.256489+0000","last_undegraded":"2024-02-05T07:50:27.256489+0000","last_fullsized":"2024-02-05T07:50:27.256489+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:56:08.567259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254062+0000","last_change":"2024-02-05T07:49:53.484201+0000","last_active":"2024-02-05T07:50:27.254062+0000","last_peered":"2024-02-05T07:50:27.254062+0000","last_clean":"2024-02-05T07:50:27.254062+0000","last_became_active":"2024-02-05T07:49:53.483871+0000","last_became_peered":"2024-02-05T07:49:53.483871+0000","last_unstale":"2024-02-05T07:50:27.254062+0000","last_undegraded":"2024-02-05T07:50:27.254062+0000","last_fullsized":"2024-02-05T07:50:27.254062+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:22:57.963594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"66'2","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359527+0000","last_change":"2024-02-05T07:49:50.453091+0000","last_active":"2024-02-05T07:50:27.359527+0000","last_peered":"2024-02-05T07:50:27.359527+0000","last_clean":"2024-02-05T07:50:27.359527+0000","last_became_active":"2024-02-05T07:49:46.429198+0000","last_became_peered":"2024-02-05T07:49:46.429198+0000","last_unstale":"2024-02-05T07:50:27.359527+0000","last_undegraded":"2024-02-05T07:50:27.359527+0000","last_fullsized":"2024-02-05T07:50:27.359527+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:45.413168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:45.413168+0000","last_clean_scrub_stamp":"2024-02-05T07:49:45.413168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:45:11.102138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018745019999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439666+0000","last_change":"2024-02-05T07:49:47.447553+0000","last_active":"2024-02-05T07:50:27.439666+0000","last_peered":"2024-02-05T07:50:27.439666+0000","last_clean":"2024-02-05T07:50:27.439666+0000","last_became_active":"2024-02-05T07:49:47.447233+0000","last_became_peered":"2024-02-05T07:49:47.447233+0000","last_unstale":"2024-02-05T07:50:27.439666+0000","last_undegraded":"2024-02-05T07:50:27.439666+0000","last_fullsized":"2024-02-05T07:50:27.439666+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:15:12.905659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"70'28","reported_seq":93,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:57.574219+0000","last_change":"2024-02-05T07:49:49.452680+0000","last_active":"2024-02-05T07:50:57.574219+0000","last_peered":"2024-02-05T07:50:57.574219+0000","last_clean":"2024-02-05T07:50:57.574219+0000","last_became_active":"2024-02-05T07:49:49.452238+0000","last_became_peered":"2024-02-05T07:49:49.452238+0000","last_unstale":"2024-02-05T07:50:57.574219+0000","last_undegraded":"2024-02-05T07:50:57.574219+0000","last_fullsized":"2024-02-05T07:50:57.574219+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:45:32.499108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.255778+0000","last_change":"2024-02-05T07:49:51.464098+0000","last_active":"2024-02-05T07:50:27.255778+0000","last_peered":"2024-02-05T07:50:27.255778+0000","last_clean":"2024-02-05T07:50:27.255778+0000","last_became_active":"2024-02-05T07:49:51.463787+0000","last_became_peered":"2024-02-05T07:49:51.463787+0000","last_unstale":"2024-02-05T07:50:27.255778+0000","last_undegraded":"2024-02-05T07:50:27.255778+0000","last_fullsized":"2024-02-05T07:50:27.255778+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:47:29.020193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.369226+0000","last_change":"2024-02-05T07:49:53.836380+0000","last_active":"2024-02-05T07:50:27.369226+0000","last_peered":"2024-02-05T07:50:27.369226+0000","last_clean":"2024-02-05T07:50:27.369226+0000","last_became_active":"2024-02-05T07:49:53.835312+0000","last_became_peered":"2024-02-05T07:49:53.835312+0000","last_unstale":"2024-02-05T07:50:27.369226+0000","last_undegraded":"2024-02-05T07:50:27.369226+0000","last_fullsized":"2024-02-05T07:50:27.369226+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:10:50.589863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333179+0000","last_change":"2024-02-05T07:49:47.519909+0000","last_active":"2024-02-05T07:50:27.333179+0000","last_peered":"2024-02-05T07:50:27.333179+0000","last_clean":"2024-02-05T07:50:27.333179+0000","last_became_active":"2024-02-05T07:49:47.519351+0000","last_became_peered":"2024-02-05T07:49:47.519351+0000","last_unstale":"2024-02-05T07:50:27.333179+0000","last_undegraded":"2024-02-05T07:50:27.333179+0000","last_fullsized":"2024-02-05T07:50:27.333179+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:14:28.389456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.364569+0000","last_change":"2024-02-05T07:49:50.449980+0000","last_active":"2024-02-05T07:50:27.364569+0000","last_peered":"2024-02-05T07:50:27.364569+0000","last_clean":"2024-02-05T07:50:27.364569+0000","last_became_active":"2024-02-05T07:49:46.427509+0000","last_became_peered":"2024-02-05T07:49:46.427509+0000","last_unstale":"2024-02-05T07:50:27.364569+0000","last_undegraded":"2024-02-05T07:50:27.364569+0000","last_fullsized":"2024-02-05T07:50:27.364569+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:45.413168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:45.413168+0000","last_clean_scrub_stamp":"2024-02-05T07:49:45.413168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:39:57.099803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042346799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"70'13","reported_seq":56,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333984+0000","last_change":"2024-02-05T07:49:49.450289+0000","last_active":"2024-02-05T07:50:27.333984+0000","last_peered":"2024-02-05T07:50:27.333984+0000","last_clean":"2024-02-05T07:50:27.333984+0000","last_became_active":"2024-02-05T07:49:49.449991+0000","last_became_peered":"2024-02-05T07:49:49.449991+0000","last_unstale":"2024-02-05T07:50:27.333984+0000","last_undegraded":"2024-02-05T07:50:27.333984+0000","last_fullsized":"2024-02-05T07:50:27.333984+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:45:17.974373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.365256+0000","last_change":"2024-02-05T07:49:51.460350+0000","last_active":"2024-02-05T07:50:27.365256+0000","last_peered":"2024-02-05T07:50:27.365256+0000","last_clean":"2024-02-05T07:50:27.365256+0000","last_became_active":"2024-02-05T07:49:51.460020+0000","last_became_peered":"2024-02-05T07:49:51.460020+0000","last_unstale":"2024-02-05T07:50:27.365256+0000","last_undegraded":"2024-02-05T07:50:27.365256+0000","last_fullsized":"2024-02-05T07:50:27.365256+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:58:49.326928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.360179+0000","last_change":"2024-02-05T07:49:53.477271+0000","last_active":"2024-02-05T07:50:27.360179+0000","last_peered":"2024-02-05T07:50:27.360179+0000","last_clean":"2024-02-05T07:50:27.360179+0000","last_became_active":"2024-02-05T07:49:53.476972+0000","last_became_peered":"2024-02-05T07:49:53.476972+0000","last_unstale":"2024-02-05T07:50:27.360179+0000","last_undegraded":"2024-02-05T07:50:27.360179+0000","last_fullsized":"2024-02-05T07:50:27.360179+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:44:05.087061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"74'128","reported_seq":279,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:29.348548+0000","last_change":"2024-02-05T07:49:37.739906+0000","last_active":"2024-02-05T07:50:29.348548+0000","last_peered":"2024-02-05T07:50:29.348548+0000","last_clean":"2024-02-05T07:50:29.348548+0000","last_became_active":"2024-02-05T07:49:37.739474+0000","last_became_peered":"2024-02-05T07:49:37.739474+0000","last_unstale":"2024-02-05T07:50:29.348548+0000","last_undegraded":"2024-02-05T07:50:29.348548+0000","last_fullsized":"2024-02-05T07:50:29.348548+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:47:29.019622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:47:29.019622+0000","last_clean_scrub_stamp":"2024-02-05T07:47:29.019622+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:05:41.765759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253652+0000","last_change":"2024-02-05T07:49:47.446478+0000","last_active":"2024-02-05T07:50:27.253652+0000","last_peered":"2024-02-05T07:50:27.253652+0000","last_clean":"2024-02-05T07:50:27.253652+0000","last_became_active":"2024-02-05T07:49:47.446211+0000","last_became_peered":"2024-02-05T07:49:47.446211+0000","last_unstale":"2024-02-05T07:50:27.253652+0000","last_undegraded":"2024-02-05T07:50:27.253652+0000","last_fullsized":"2024-02-05T07:50:27.253652+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:48:08.108256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"70'16","reported_seq":67,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:57.574491+0000","last_change":"2024-02-05T07:49:49.735901+0000","last_active":"2024-02-05T07:50:57.574491+0000","last_peered":"2024-02-05T07:50:57.574491+0000","last_clean":"2024-02-05T07:50:57.574491+0000","last_became_active":"2024-02-05T07:49:49.735541+0000","last_became_peered":"2024-02-05T07:49:49.735541+0000","last_unstale":"2024-02-05T07:50:57.574491+0000","last_undegraded":"2024-02-05T07:50:57.574491+0000","last_fullsized":"2024-02-05T07:50:57.574491+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:45:11.927584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368967+0000","last_change":"2024-02-05T07:49:51.462683+0000","last_active":"2024-02-05T07:50:27.368967+0000","last_peered":"2024-02-05T07:50:27.368967+0000","last_clean":"2024-02-05T07:50:27.368967+0000","last_became_active":"2024-02-05T07:49:51.462419+0000","last_became_peered":"2024-02-05T07:49:51.462419+0000","last_unstale":"2024-02-05T07:50:27.368967+0000","last_undegraded":"2024-02-05T07:50:27.368967+0000","last_fullsized":"2024-02-05T07:50:27.368967+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:10:54.226467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333275+0000","last_change":"2024-02-05T07:49:53.481952+0000","last_active":"2024-02-05T07:50:27.333275+0000","last_peered":"2024-02-05T07:50:27.333275+0000","last_clean":"2024-02-05T07:50:27.333275+0000","last_became_active":"2024-02-05T07:49:53.481621+0000","last_became_peered":"2024-02-05T07:49:53.481621+0000","last_unstale":"2024-02-05T07:50:27.333275+0000","last_undegraded":"2024-02-05T07:50:27.333275+0000","last_fullsized":"2024-02-05T07:50:27.333275+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:40:05.915673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.255115+0000","last_change":"2024-02-05T07:49:47.449219+0000","last_active":"2024-02-05T07:50:27.255115+0000","last_peered":"2024-02-05T07:50:27.255115+0000","last_clean":"2024-02-05T07:50:27.255115+0000","last_became_active":"2024-02-05T07:49:47.448931+0000","last_became_peered":"2024-02-05T07:49:47.448931+0000","last_unstale":"2024-02-05T07:50:27.255115+0000","last_undegraded":"2024-02-05T07:50:27.255115+0000","last_fullsized":"2024-02-05T07:50:27.255115+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:20:04.989220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"71'5","reported_seq":114,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:51:02.221838+0000","last_change":"2024-02-05T07:49:50.456166+0000","last_active":"2024-02-05T07:51:02.221838+0000","last_peered":"2024-02-05T07:51:02.221838+0000","last_clean":"2024-02-05T07:51:02.221838+0000","last_became_active":"2024-02-05T07:49:46.433572+0000","last_became_peered":"2024-02-05T07:49:46.433572+0000","last_unstale":"2024-02-05T07:51:02.221838+0000","last_undegraded":"2024-02-05T07:51:02.221838+0000","last_fullsized":"2024-02-05T07:51:02.221838+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:45.413168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:45.413168+0000","last_clean_scrub_stamp":"2024-02-05T07:49:45.413168+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:51:34.023397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015152,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":69,"num_read_kb":64,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"70'12","reported_seq":47,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256393+0000","last_change":"2024-02-05T07:49:49.447760+0000","last_active":"2024-02-05T07:50:27.256393+0000","last_peered":"2024-02-05T07:50:27.256393+0000","last_clean":"2024-02-05T07:50:27.256393+0000","last_became_active":"2024-02-05T07:49:49.447432+0000","last_became_peered":"2024-02-05T07:49:49.447432+0000","last_unstale":"2024-02-05T07:50:27.256393+0000","last_undegraded":"2024-02-05T07:50:27.256393+0000","last_fullsized":"2024-02-05T07:50:27.256393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:56:25.889695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.360392+0000","last_change":"2024-02-05T07:49:51.468986+0000","last_active":"2024-02-05T07:50:27.360392+0000","last_peered":"2024-02-05T07:50:27.360392+0000","last_clean":"2024-02-05T07:50:27.360392+0000","last_became_active":"2024-02-05T07:49:51.468720+0000","last_became_peered":"2024-02-05T07:49:51.468720+0000","last_unstale":"2024-02-05T07:50:27.360392+0000","last_undegraded":"2024-02-05T07:50:27.360392+0000","last_fullsized":"2024-02-05T07:50:27.360392+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:11:27.089767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333044+0000","last_change":"2024-02-05T07:49:53.835527+0000","last_active":"2024-02-05T07:50:27.333044+0000","last_peered":"2024-02-05T07:50:27.333044+0000","last_clean":"2024-02-05T07:50:27.333044+0000","last_became_active":"2024-02-05T07:49:53.834607+0000","last_became_peered":"2024-02-05T07:49:53.834607+0000","last_unstale":"2024-02-05T07:50:27.333044+0000","last_undegraded":"2024-02-05T07:50:27.333044+0000","last_fullsized":"2024-02-05T07:50:27.333044+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:57:38.229214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"63'1","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333727+0000","last_change":"2024-02-05T07:49:47.519808+0000","last_active":"2024-02-05T07:50:27.333727+0000","last_peered":"2024-02-05T07:50:27.333727+0000","last_clean":"2024-02-05T07:50:27.333727+0000","last_became_active":"2024-02-05T07:49:47.519528+0000","last_became_peered":"2024-02-05T07:49:47.519528+0000","last_unstale":"2024-02-05T07:50:27.333727+0000","last_undegraded":"2024-02-05T07:50:27.333727+0000","last_fullsized":"2024-02-05T07:50:27.333727+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:19:29.206520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"70'19","reported_seq":65,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256663+0000","last_change":"2024-02-05T07:49:49.734608+0000","last_active":"2024-02-05T07:50:27.256663+0000","last_peered":"2024-02-05T07:50:27.256663+0000","last_clean":"2024-02-05T07:50:27.256663+0000","last_became_active":"2024-02-05T07:49:49.734296+0000","last_became_peered":"2024-02-05T07:49:49.734296+0000","last_unstale":"2024-02-05T07:50:27.256663+0000","last_undegraded":"2024-02-05T07:50:27.256663+0000","last_fullsized":"2024-02-05T07:50:27.256663+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:16:40.799503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253379+0000","last_change":"2024-02-05T07:49:51.464033+0000","last_active":"2024-02-05T07:50:27.253379+0000","last_peered":"2024-02-05T07:50:27.253379+0000","last_clean":"2024-02-05T07:50:27.253379+0000","last_became_active":"2024-02-05T07:49:51.463720+0000","last_became_peered":"2024-02-05T07:49:51.463720+0000","last_unstale":"2024-02-05T07:50:27.253379+0000","last_undegraded":"2024-02-05T07:50:27.253379+0000","last_fullsized":"2024-02-05T07:50:27.253379+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:45:44.086697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254382+0000","last_change":"2024-02-05T07:49:53.483687+0000","last_active":"2024-02-05T07:50:27.254382+0000","last_peered":"2024-02-05T07:50:27.254382+0000","last_clean":"2024-02-05T07:50:27.254382+0000","last_became_active":"2024-02-05T07:49:53.483407+0000","last_became_peered":"2024-02-05T07:49:53.483407+0000","last_unstale":"2024-02-05T07:50:27.254382+0000","last_undegraded":"2024-02-05T07:50:27.254382+0000","last_fullsized":"2024-02-05T07:50:27.254382+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:00:31.813086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.360071+0000","last_change":"2024-02-05T07:49:47.519813+0000","last_active":"2024-02-05T07:50:27.360071+0000","last_peered":"2024-02-05T07:50:27.360071+0000","last_clean":"2024-02-05T07:50:27.360071+0000","last_became_active":"2024-02-05T07:49:47.519527+0000","last_became_peered":"2024-02-05T07:49:47.519527+0000","last_unstale":"2024-02-05T07:50:27.360071+0000","last_undegraded":"2024-02-05T07:50:27.360071+0000","last_fullsized":"2024-02-05T07:50:27.360071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:54:09.621261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"70'10","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.364466+0000","last_change":"2024-02-05T07:50:11.867536+0000","last_active":"2024-02-05T07:50:27.364466+0000","last_peered":"2024-02-05T07:50:27.364466+0000","last_clean":"2024-02-05T07:50:27.364466+0000","last_became_active":"2024-02-05T07:50:11.859627+0000","last_became_peered":"2024-02-05T07:50:11.859627+0000","last_unstale":"2024-02-05T07:50:27.364466+0000","last_undegraded":"2024-02-05T07:50:27.364466+0000","last_fullsized":"2024-02-05T07:50:27.364466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:22:14.912788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:57.574216+0000","last_change":"2024-02-05T07:49:51.462401+0000","last_active":"2024-02-05T07:50:57.574216+0000","last_peered":"2024-02-05T07:50:57.574216+0000","last_clean":"2024-02-05T07:50:57.574216+0000","last_became_active":"2024-02-05T07:49:51.461018+0000","last_became_peered":"2024-02-05T07:49:51.461018+0000","last_unstale":"2024-02-05T07:50:57.574216+0000","last_undegraded":"2024-02-05T07:50:57.574216+0000","last_fullsized":"2024-02-05T07:50:57.574216+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:25:38.200242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368836+0000","last_change":"2024-02-05T07:49:53.836234+0000","last_active":"2024-02-05T07:50:27.368836+0000","last_peered":"2024-02-05T07:50:27.368836+0000","last_clean":"2024-02-05T07:50:27.368836+0000","last_became_active":"2024-02-05T07:49:53.835945+0000","last_became_peered":"2024-02-05T07:49:53.835945+0000","last_unstale":"2024-02-05T07:50:27.368836+0000","last_undegraded":"2024-02-05T07:50:27.368836+0000","last_fullsized":"2024-02-05T07:50:27.368836+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:05:54.982258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"63'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256514+0000","last_change":"2024-02-05T07:49:47.445692+0000","last_active":"2024-02-05T07:50:27.256514+0000","last_peered":"2024-02-05T07:50:27.256514+0000","last_clean":"2024-02-05T07:50:27.256514+0000","last_became_active":"2024-02-05T07:49:47.445365+0000","last_became_peered":"2024-02-05T07:49:47.445365+0000","last_unstale":"2024-02-05T07:50:27.256514+0000","last_undegraded":"2024-02-05T07:50:27.256514+0000","last_fullsized":"2024-02-05T07:50:27.256514+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:05:47.500048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"70'14","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439367+0000","last_change":"2024-02-05T07:49:49.442410+0000","last_active":"2024-02-05T07:50:27.439367+0000","last_peered":"2024-02-05T07:50:27.439367+0000","last_clean":"2024-02-05T07:50:27.439367+0000","last_became_active":"2024-02-05T07:49:49.441931+0000","last_became_peered":"2024-02-05T07:49:49.441931+0000","last_unstale":"2024-02-05T07:50:27.439367+0000","last_undegraded":"2024-02-05T07:50:27.439367+0000","last_fullsized":"2024-02-05T07:50:27.439367+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:59:10.646362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"70'8","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:57.574210+0000","last_change":"2024-02-05T07:49:51.467071+0000","last_active":"2024-02-05T07:50:57.574210+0000","last_peered":"2024-02-05T07:50:57.574210+0000","last_clean":"2024-02-05T07:50:57.574210+0000","last_became_active":"2024-02-05T07:49:51.466715+0000","last_became_peered":"2024-02-05T07:49:51.466715+0000","last_unstale":"2024-02-05T07:50:57.574210+0000","last_undegraded":"2024-02-05T07:50:57.574210+0000","last_fullsized":"2024-02-05T07:50:57.574210+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:32:35.577834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439272+0000","last_change":"2024-02-05T07:49:53.479631+0000","last_active":"2024-02-05T07:50:27.439272+0000","last_peered":"2024-02-05T07:50:27.439272+0000","last_clean":"2024-02-05T07:50:27.439272+0000","last_became_active":"2024-02-05T07:49:53.479347+0000","last_became_peered":"2024-02-05T07:49:53.479347+0000","last_unstale":"2024-02-05T07:50:27.439272+0000","last_undegraded":"2024-02-05T07:50:27.439272+0000","last_fullsized":"2024-02-05T07:50:27.439272+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:42:12.523750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254293+0000","last_change":"2024-02-05T07:49:47.440757+0000","last_active":"2024-02-05T07:50:27.254293+0000","last_peered":"2024-02-05T07:50:27.254293+0000","last_clean":"2024-02-05T07:50:27.254293+0000","last_became_active":"2024-02-05T07:49:47.440474+0000","last_became_peered":"2024-02-05T07:49:47.440474+0000","last_unstale":"2024-02-05T07:50:27.254293+0000","last_undegraded":"2024-02-05T07:50:27.254293+0000","last_fullsized":"2024-02-05T07:50:27.254293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:54:48.519955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"70'18","reported_seq":61,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252421+0000","last_change":"2024-02-05T07:49:49.736372+0000","last_active":"2024-02-05T07:50:27.252421+0000","last_peered":"2024-02-05T07:50:27.252421+0000","last_clean":"2024-02-05T07:50:27.252421+0000","last_became_active":"2024-02-05T07:49:49.735956+0000","last_became_peered":"2024-02-05T07:49:49.735956+0000","last_unstale":"2024-02-05T07:50:27.252421+0000","last_undegraded":"2024-02-05T07:50:27.252421+0000","last_fullsized":"2024-02-05T07:50:27.252421+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:42:41.997513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439194+0000","last_change":"2024-02-05T07:49:51.465286+0000","last_active":"2024-02-05T07:50:27.439194+0000","last_peered":"2024-02-05T07:50:27.439194+0000","last_clean":"2024-02-05T07:50:27.439194+0000","last_became_active":"2024-02-05T07:49:51.464861+0000","last_became_peered":"2024-02-05T07:49:51.464861+0000","last_unstale":"2024-02-05T07:50:27.439194+0000","last_undegraded":"2024-02-05T07:50:27.439194+0000","last_fullsized":"2024-02-05T07:50:27.439194+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:50:08.659066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359774+0000","last_change":"2024-02-05T07:49:53.476564+0000","last_active":"2024-02-05T07:50:27.359774+0000","last_peered":"2024-02-05T07:50:27.359774+0000","last_clean":"2024-02-05T07:50:27.359774+0000","last_became_active":"2024-02-05T07:49:53.476286+0000","last_became_peered":"2024-02-05T07:49:53.476286+0000","last_unstale":"2024-02-05T07:50:27.359774+0000","last_undegraded":"2024-02-05T07:50:27.359774+0000","last_fullsized":"2024-02-05T07:50:27.359774+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:48:34.681644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:57.574558+0000","last_change":"2024-02-05T07:49:51.462755+0000","last_active":"2024-02-05T07:50:57.574558+0000","last_peered":"2024-02-05T07:50:57.574558+0000","last_clean":"2024-02-05T07:50:57.574558+0000","last_became_active":"2024-02-05T07:49:51.462362+0000","last_became_peered":"2024-02-05T07:49:51.462362+0000","last_unstale":"2024-02-05T07:50:57.574558+0000","last_undegraded":"2024-02-05T07:50:57.574558+0000","last_fullsized":"2024-02-05T07:50:57.574558+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:36:47.969011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252336+0000","last_change":"2024-02-05T07:49:47.443430+0000","last_active":"2024-02-05T07:50:27.252336+0000","last_peered":"2024-02-05T07:50:27.252336+0000","last_clean":"2024-02-05T07:50:27.252336+0000","last_became_active":"2024-02-05T07:49:47.443157+0000","last_became_peered":"2024-02-05T07:49:47.443157+0000","last_unstale":"2024-02-05T07:50:27.252336+0000","last_undegraded":"2024-02-05T07:50:27.252336+0000","last_fullsized":"2024-02-05T07:50:27.252336+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:35:21.792177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333952+0000","last_change":"2024-02-05T07:49:49.452089+0000","last_active":"2024-02-05T07:50:27.333952+0000","last_peered":"2024-02-05T07:50:27.333952+0000","last_clean":"2024-02-05T07:50:27.333952+0000","last_became_active":"2024-02-05T07:49:49.451704+0000","last_became_peered":"2024-02-05T07:49:49.451704+0000","last_unstale":"2024-02-05T07:50:27.333952+0000","last_undegraded":"2024-02-05T07:50:27.333952+0000","last_fullsized":"2024-02-05T07:50:27.333952+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:35:06.578768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252463+0000","last_change":"2024-02-05T07:49:53.835233+0000","last_active":"2024-02-05T07:50:27.252463+0000","last_peered":"2024-02-05T07:50:27.252463+0000","last_clean":"2024-02-05T07:50:27.252463+0000","last_became_active":"2024-02-05T07:49:53.834847+0000","last_became_peered":"2024-02-05T07:49:53.834847+0000","last_unstale":"2024-02-05T07:50:27.252463+0000","last_undegraded":"2024-02-05T07:50:27.252463+0000","last_fullsized":"2024-02-05T07:50:27.252463+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:17:06.871213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.360355+0000","last_change":"2024-02-05T07:49:51.466399+0000","last_active":"2024-02-05T07:50:27.360355+0000","last_peered":"2024-02-05T07:50:27.360355+0000","last_clean":"2024-02-05T07:50:27.360355+0000","last_became_active":"2024-02-05T07:49:51.466099+0000","last_became_peered":"2024-02-05T07:49:51.466099+0000","last_unstale":"2024-02-05T07:50:27.360355+0000","last_undegraded":"2024-02-05T07:50:27.360355+0000","last_fullsized":"2024-02-05T07:50:27.360355+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:43:11.141949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439260+0000","last_change":"2024-02-05T07:49:47.519778+0000","last_active":"2024-02-05T07:50:27.439260+0000","last_peered":"2024-02-05T07:50:27.439260+0000","last_clean":"2024-02-05T07:50:27.439260+0000","last_became_active":"2024-02-05T07:49:47.519469+0000","last_became_peered":"2024-02-05T07:49:47.519469+0000","last_unstale":"2024-02-05T07:50:27.439260+0000","last_undegraded":"2024-02-05T07:50:27.439260+0000","last_fullsized":"2024-02-05T07:50:27.439260+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:37:24.792568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"70'11","reported_seq":66,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.440225+0000","last_change":"2024-02-05T07:50:11.866569+0000","last_active":"2024-02-05T07:50:27.440225+0000","last_peered":"2024-02-05T07:50:27.440225+0000","last_clean":"2024-02-05T07:50:27.440225+0000","last_became_active":"2024-02-05T07:50:11.857881+0000","last_became_peered":"2024-02-05T07:50:11.857881+0000","last_unstale":"2024-02-05T07:50:27.440225+0000","last_undegraded":"2024-02-05T07:50:27.440225+0000","last_fullsized":"2024-02-05T07:50:27.440225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:27:32.387706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.365277+0000","last_change":"2024-02-05T07:49:53.485182+0000","last_active":"2024-02-05T07:50:27.365277+0000","last_peered":"2024-02-05T07:50:27.365277+0000","last_clean":"2024-02-05T07:50:27.365277+0000","last_became_active":"2024-02-05T07:49:53.484590+0000","last_became_peered":"2024-02-05T07:49:53.484590+0000","last_unstale":"2024-02-05T07:50:27.365277+0000","last_undegraded":"2024-02-05T07:50:27.365277+0000","last_fullsized":"2024-02-05T07:50:27.365277+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:19:06.287158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333840+0000","last_change":"2024-02-05T07:49:51.468829+0000","last_active":"2024-02-05T07:50:27.333840+0000","last_peered":"2024-02-05T07:50:27.333840+0000","last_clean":"2024-02-05T07:50:27.333840+0000","last_became_active":"2024-02-05T07:49:51.468495+0000","last_became_peered":"2024-02-05T07:49:51.468495+0000","last_unstale":"2024-02-05T07:50:27.333840+0000","last_undegraded":"2024-02-05T07:50:27.333840+0000","last_fullsized":"2024-02-05T07:50:27.333840+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:50:32.323195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253271+0000","last_change":"2024-02-05T07:49:47.449403+0000","last_active":"2024-02-05T07:50:27.253271+0000","last_peered":"2024-02-05T07:50:27.253271+0000","last_clean":"2024-02-05T07:50:27.253271+0000","last_became_active":"2024-02-05T07:49:47.448970+0000","last_became_peered":"2024-02-05T07:49:47.448970+0000","last_unstale":"2024-02-05T07:50:27.253271+0000","last_undegraded":"2024-02-05T07:50:27.253271+0000","last_fullsized":"2024-02-05T07:50:27.253271+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:33:28.806519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"70'17","reported_seq":57,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.440139+0000","last_change":"2024-02-05T07:49:49.446195+0000","last_active":"2024-02-05T07:50:27.440139+0000","last_peered":"2024-02-05T07:50:27.440139+0000","last_clean":"2024-02-05T07:50:27.440139+0000","last_became_active":"2024-02-05T07:49:49.445922+0000","last_became_peered":"2024-02-05T07:49:49.445922+0000","last_unstale":"2024-02-05T07:50:27.440139+0000","last_undegraded":"2024-02-05T07:50:27.440139+0000","last_fullsized":"2024-02-05T07:50:27.440139+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:03:12.272146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.440379+0000","last_change":"2024-02-05T07:49:53.480356+0000","last_active":"2024-02-05T07:50:27.440379+0000","last_peered":"2024-02-05T07:50:27.440379+0000","last_clean":"2024-02-05T07:50:27.440379+0000","last_became_active":"2024-02-05T07:49:53.480089+0000","last_became_peered":"2024-02-05T07:49:53.480089+0000","last_unstale":"2024-02-05T07:50:27.440379+0000","last_undegraded":"2024-02-05T07:50:27.440379+0000","last_fullsized":"2024-02-05T07:50:27.440379+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:17:16.231336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:57.574220+0000","last_change":"2024-02-05T07:49:51.461674+0000","last_active":"2024-02-05T07:50:57.574220+0000","last_peered":"2024-02-05T07:50:57.574220+0000","last_clean":"2024-02-05T07:50:57.574220+0000","last_became_active":"2024-02-05T07:49:51.461393+0000","last_became_peered":"2024-02-05T07:49:51.461393+0000","last_unstale":"2024-02-05T07:50:57.574220+0000","last_undegraded":"2024-02-05T07:50:57.574220+0000","last_fullsized":"2024-02-05T07:50:57.574220+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:22:33.553443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253933+0000","last_change":"2024-02-05T07:49:47.447353+0000","last_active":"2024-02-05T07:50:27.253933+0000","last_peered":"2024-02-05T07:50:27.253933+0000","last_clean":"2024-02-05T07:50:27.253933+0000","last_became_active":"2024-02-05T07:49:47.447006+0000","last_became_peered":"2024-02-05T07:49:47.447006+0000","last_unstale":"2024-02-05T07:50:27.253933+0000","last_undegraded":"2024-02-05T07:50:27.253933+0000","last_fullsized":"2024-02-05T07:50:27.253933+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:11:03.724451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439903+0000","last_change":"2024-02-05T07:49:49.451678+0000","last_active":"2024-02-05T07:50:27.439903+0000","last_peered":"2024-02-05T07:50:27.439903+0000","last_clean":"2024-02-05T07:50:27.439903+0000","last_became_active":"2024-02-05T07:49:49.451264+0000","last_became_peered":"2024-02-05T07:49:49.451264+0000","last_unstale":"2024-02-05T07:50:27.439903+0000","last_undegraded":"2024-02-05T07:50:27.439903+0000","last_fullsized":"2024-02-05T07:50:27.439903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:13:15.797062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":64,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":56,"seq":240518168596,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712676,"statfs":{"total":95995035648,"available":95961780224,"internally_reserved":0,"allocated":1327104,"data_stored":878713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.30099999999999999}]},{"osd":1,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.46899999999999997}]},{"osd":2,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.32100000000000001}]},{"osd":3,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.40799999999999997}]},{"osd":4,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.42199999999999999}]},{"osd":5,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.436}]},{"osd":6,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.44700000000000001}]}]},{"osd":6,"up_from":47,"seq":201863462937,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32456,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712696,"statfs":{"total":95995035648,"available":95961800704,"internally_reserved":0,"allocated":1310720,"data_stored":877675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.35699999999999998}]},{"osd":1,"last update":"Mon Feb 5 07:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.64900000000000002}]},{"osd":2,"last update":"Mon Feb 5 07:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.53100000000000003}]},{"osd":3,"last update":"Mon Feb 5 07:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.629},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.60899999999999999}]},{"osd":4,"last update":"Mon Feb 5 07:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.55500000000000005}]},{"osd":5,"last update":"Mon Feb 5 07:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.72199999999999998}]},{"osd":7,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.70799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27780,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717372,"statfs":{"total":95995035648,"available":95966588928,"internally_reserved":0,"allocated":778240,"data_stored":289367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.34399999999999997,"15min":0.31900000000000001},"min":{"1min":0.23300000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.36599999999999999,"15min":0.33100000000000002},"min":{"1min":0.28599999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.34599999999999997}]},{"osd":2,"last update":"Mon Feb 5 07:50:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.40500000000000003},"min":{"1min":0.30599999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60999999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.41499999999999998,"15min":0.40300000000000002},"min":{"1min":0.28699999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.31900000000000001}]},{"osd":3,"last update":"Mon Feb 5 07:50:56 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59099999999999997,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.42599999999999999,"15min":0.436},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.56699999999999995,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.33400000000000002}]},{"osd":4,"last update":"Mon Feb 5 07:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57199999999999995,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67100000000000004,"5min":0.751,"15min":0.751},"last":0.625},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.74199999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58599999999999997}]},{"osd":5,"last update":"Mon Feb 5 07:50:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.71799999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.746}]},{"osd":6,"last update":"Mon Feb 5 07:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.66300000000000003}]},{"osd":7,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.79700000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28324,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716828,"statfs":{"total":95995035648,"available":95966031872,"internally_reserved":0,"allocated":1339392,"data_stored":879343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 5 07:50:04 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.33200000000000002},"min":{"1min":0.24399999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.372,"15min":0.34399999999999997},"min":{"1min":0.317,"5min":0.223,"15min":0.223},"max":{"1min":0.63600000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.27900000000000003}]},{"osd":2,"last update":"Mon Feb 5 07:50:41 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40699999999999997,"15min":0.40600000000000003},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54600000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.39500000000000002,"15min":0.38500000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.54900000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.38200000000000001}]},{"osd":3,"last update":"Mon Feb 5 07:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.47499999999999998,"15min":0.49099999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.56799999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.371},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.46400000000000002,"15min":0.48099999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.58499999999999996,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.442}]},{"osd":4,"last update":"Mon Feb 5 07:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.749,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.746,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.58699999999999997}]},{"osd":5,"last update":"Mon Feb 5 07:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.65900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57399999999999995,"15min":0.58199999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.74299999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.61699999999999999}]},{"osd":6,"last update":"Mon Feb 5 07:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.60499999999999998}]},{"osd":7,"last update":"Mon Feb 5 07:50:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.628}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717432,"statfs":{"total":95995035648,"available":95966650368,"internally_reserved":0,"allocated":716800,"data_stored":287916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.38400000000000001,"15min":0.374},"min":{"1min":0.312,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.39400000000000002},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.253,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.40999999999999998,"15min":0.40799999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.30599999999999999}]},{"osd":3,"last update":"Mon Feb 5 07:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46100000000000002,"15min":0.46899999999999997},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.38},{"interface":"front","average":{"1min":0.432,"5min":0.46400000000000002,"15min":0.47199999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.34499999999999997}]},{"osd":4,"last update":"Mon Feb 5 07:50:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.46000000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.496}]},{"osd":5,"last update":"Mon Feb 5 07:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54800000000000004,"15min":0.55100000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46500000000000002}]},{"osd":6,"last update":"Mon Feb 5 07:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.44800000000000001}]},{"osd":7,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.53500000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149737,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31916,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713236,"statfs":{"total":95995035648,"available":95962353664,"internally_reserved":0,"allocated":753664,"data_stored":289059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.41299999999999998,"15min":0.42899999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.442,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40899999999999997,"15min":0.41899999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.47699999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.308}]},{"osd":1,"last update":"Mon Feb 5 07:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40999999999999998,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.51200000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.45100000000000001,"15min":0.45500000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.52000000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.36099999999999999}]},{"osd":2,"last update":"Mon Feb 5 07:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.46400000000000002,"15min":0.47499999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65700000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.436,"15min":0.45000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53700000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.34899999999999998}]},{"osd":4,"last update":"Mon Feb 5 07:50:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80500000000000005,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.81999999999999995,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54500000000000004}]},{"osd":5,"last update":"Mon Feb 5 07:50:42 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.82799999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53400000000000003}]},{"osd":6,"last update":"Mon Feb 5 07:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.47699999999999998}]},{"osd":7,"last update":"Mon Feb 5 07:50:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.58399999999999996}]}]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27744,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717408,"statfs":{"total":95995035648,"available":95966625792,"internally_reserved":0,"allocated":741376,"data_stored":287899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79700000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.71499999999999997,"5min":0.879,"15min":0.879},"last":0.63100000000000001}]},{"osd":1,"last update":"Mon Feb 5 07:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82399999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.51600000000000001}]},{"osd":2,"last update":"Mon Feb 5 07:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.56200000000000006,"15min":0.57099999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65800000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.59499999999999997}]},{"osd":3,"last update":"Mon Feb 5 07:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.903,"5min":2.903,"15min":2.903},"last":0.379},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.42199999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78600000000000003,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.437}]},{"osd":5,"last update":"Mon Feb 5 07:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.439,"15min":0.42999999999999999},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.46200000000000002,"15min":0.45200000000000001},"min":{"1min":0.35399999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.66300000000000003}]},{"osd":6,"last update":"Mon Feb 5 07:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.67600000000000005}]},{"osd":7,"last update":"Mon Feb 5 07:50:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.54200000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724575,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31892,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713260,"statfs":{"total":95995035648,"available":95962378240,"internally_reserved":0,"allocated":729088,"data_stored":288904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.40899999999999997}]},{"osd":1,"last update":"Mon Feb 5 07:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.53100000000000003}]},{"osd":2,"last update":"Mon Feb 5 07:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.373,"5min":0.372,"15min":0.372},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.60199999999999998}]},{"osd":3,"last update":"Mon Feb 5 07:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.40600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.42099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.67200000000000004,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.66600000000000004}]},{"osd":4,"last update":"Mon Feb 5 07:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.40899999999999997,"15min":0.40200000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.501},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39000000000000001,"15min":0.38700000000000001},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63100000000000001,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.63400000000000001}]},{"osd":6,"last update":"Mon Feb 5 07:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.623},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.64900000000000002}]},{"osd":7,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-05T07:51:04.992 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph pg dump --format=json 2024-02-05T07:51:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:04 smithi157 ceph-mon[33345]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:04 smithi157 ceph-mon[33345]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:51:05.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:04 smithi131 ceph-mon[41654]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:05.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:04 smithi131 ceph-mon[41654]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:51:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:04 smithi131 ceph-mon[29507]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:04 smithi131 ceph-mon[29507]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:51:05.359 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:51:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:06 smithi157 ceph-mon[33345]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:06 smithi131 ceph-mon[41654]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:06 smithi131 ceph-mon[29507]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:08.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:51:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:51:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.080+0000 7fea75553700 1 -- 172.21.15.131:0/519065289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea701008c0 msgr2=0x7fea70100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.080+0000 7fea75553700 1 --2- 172.21.15.131:0/519065289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea701008c0 0x7fea70100ca0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fea68009990 tx=0x7fea68009ca0 comp rx=0 tx=0).stop 2024-02-05T07:51:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.080+0000 7fea75553700 1 -- 172.21.15.131:0/519065289 shutdown_connections 2024-02-05T07:51:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.080+0000 7fea75553700 1 --2- 172.21.15.131:0/519065289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea701008c0 0x7fea70100ca0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.080+0000 7fea75553700 1 --2- 172.21.15.131:0/519065289 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fea701070e0 0x7fea7010ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.080+0000 7fea75553700 1 --2- 172.21.15.131:0/519065289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea70101270 0x7fea701069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.080+0000 7fea75553700 1 -- 172.21.15.131:0/519065289 >> 172.21.15.131:0/519065289 conn(0x7fea70075f80 msgr2=0x7fea70076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:51:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.080+0000 7fea75553700 1 -- 172.21.15.131:0/519065289 shutdown_connections 2024-02-05T07:51:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.080+0000 7fea75553700 1 -- 172.21.15.131:0/519065289 wait complete. 2024-02-05T07:51:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.080+0000 7fea75553700 1 Processor -- start 2024-02-05T07:51:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea75553700 1 -- start start 2024-02-05T07:51:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea75553700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fea701008c0 0x7fea70101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:51:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea75553700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea70101270 0x7fea70101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:51:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea75553700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea701070e0 0x7fea701022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:51:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea75553700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea701a5d20 con 0x7fea70101270 2024-02-05T07:51:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea75553700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea701a5e90 con 0x7fea701070e0 2024-02-05T07:51:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea75553700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea701a6050 con 0x7fea701008c0 2024-02-05T07:51:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea6e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea70101270 0x7fea70101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:51:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea6e7fc700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea70101270 0x7fea70101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39662/0 (socket says 172.21.15.131:39662) 2024-02-05T07:51:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea6e7fc700 1 -- 172.21.15.131:0/422280364 learned_addr learned my addr 172.21.15.131:0/422280364 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:51:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea6effd700 1 --2- 172.21.15.131:0/422280364 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fea701008c0 0x7fea70101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:51:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea6f7fe700 1 --2- 172.21.15.131:0/422280364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea701070e0 0x7fea701022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:51:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea6e7fc700 1 -- 172.21.15.131:0/422280364 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fea701008c0 msgr2=0x7fea70101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea6e7fc700 1 --2- 172.21.15.131:0/422280364 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fea701008c0 0x7fea70101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:09.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea6e7fc700 1 -- 172.21.15.131:0/422280364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea701070e0 msgr2=0x7fea701022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea6e7fc700 1 --2- 172.21.15.131:0/422280364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea701070e0 0x7fea701022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea6e7fc700 1 -- 172.21.15.131:0/422280364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea60009c40 con 0x7fea70101270 2024-02-05T07:51:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea6f7fe700 1 --2- 172.21.15.131:0/422280364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea701070e0 0x7fea701022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T07:51:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea6effd700 1 --2- 172.21.15.131:0/422280364 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fea701008c0 0x7fea70101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:51:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea6e7fc700 1 --2- 172.21.15.131:0/422280364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea70101270 0x7fea70101d80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea5800c9b0 tx=0x7fea5800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:51:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.081+0000 7fea67fff700 1 -- 172.21.15.131:0/422280364 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58007aa0 con 0x7fea70101270 2024-02-05T07:51:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.082+0000 7fea67fff700 1 -- 172.21.15.131:0/422280364 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fea58007c00 con 0x7fea70101270 2024-02-05T07:51:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.082+0000 7fea75553700 1 -- 172.21.15.131:0/422280364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea680095f0 con 0x7fea70101270 2024-02-05T07:51:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.082+0000 7fea67fff700 1 -- 172.21.15.131:0/422280364 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58018660 con 0x7fea70101270 2024-02-05T07:51:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.082+0000 7fea75553700 1 -- 172.21.15.131:0/422280364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea701a66a0 con 0x7fea70101270 2024-02-05T07:51:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.083+0000 7fea75553700 1 -- 172.21.15.131:0/422280364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7010a340 con 0x7fea70101270 2024-02-05T07:51:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.083+0000 7fea67fff700 1 -- 172.21.15.131:0/422280364 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fea5800f450 con 0x7fea70101270 2024-02-05T07:51:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.083+0000 7fea67fff700 1 --2- 172.21.15.131:0/422280364 >> v2:172.21.15.131:6800/2614238408 conn(0x7fea5c06da80 0x7fea5c06ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:51:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.083+0000 7fea6effd700 1 --2- 172.21.15.131:0/422280364 >> v2:172.21.15.131:6800/2614238408 conn(0x7fea5c06da80 0x7fea5c06ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:51:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.084+0000 7fea67fff700 1 -- 172.21.15.131:0/422280364 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7fea5808d450 con 0x7fea70101270 2024-02-05T07:51:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.084+0000 7fea6effd700 1 --2- 172.21.15.131:0/422280364 >> v2:172.21.15.131:6800/2614238408 conn(0x7fea5c06da80 0x7fea5c06ff40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea680098e0 tx=0x7fea68005960 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:51:09.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.088+0000 7fea67fff700 1 -- 172.21.15.131:0/422280364 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fea5805a920 con 0x7fea70101270 2024-02-05T07:51:09.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:08 smithi157 ceph-mon[33345]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:09.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:08 smithi131 ceph-mon[41654]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:09.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:08 smithi131 ceph-mon[29507]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:09.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.281+0000 7fea75553700 1 -- 172.21.15.131:0/422280364 --> v2:172.21.15.131:6800/2614238408 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fea70102d70 con 0x7fea5c06da80 2024-02-05T07:51:09.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.292+0000 7fea67fff700 1 -- 172.21.15.131:0/422280364 <== mgr.14571 v2:172.21.15.131:6800/2614238408 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+384447 (secure 0 0 0) 0x7fea70102d70 con 0x7fea5c06da80 2024-02-05T07:51:09.299 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:51:09.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.297+0000 7fea75553700 1 -- 172.21.15.131:0/422280364 >> v2:172.21.15.131:6800/2614238408 conn(0x7fea5c06da80 msgr2=0x7fea5c06ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:09.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.297+0000 7fea75553700 1 --2- 172.21.15.131:0/422280364 >> v2:172.21.15.131:6800/2614238408 conn(0x7fea5c06da80 0x7fea5c06ff40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea680098e0 tx=0x7fea68005960 comp rx=0 tx=0).stop 2024-02-05T07:51:09.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.297+0000 7fea75553700 1 -- 172.21.15.131:0/422280364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea70101270 msgr2=0x7fea70101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:09.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.297+0000 7fea75553700 1 --2- 172.21.15.131:0/422280364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea70101270 0x7fea70101d80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea5800c9b0 tx=0x7fea5800ccc0 comp rx=0 tx=0).stop 2024-02-05T07:51:09.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.297+0000 7fea75553700 1 -- 172.21.15.131:0/422280364 shutdown_connections 2024-02-05T07:51:09.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.297+0000 7fea75553700 1 --2- 172.21.15.131:0/422280364 >> v2:172.21.15.131:6800/2614238408 conn(0x7fea5c06da80 0x7fea5c06ff40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:09.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.297+0000 7fea75553700 1 --2- 172.21.15.131:0/422280364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea701070e0 0x7fea701022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:09.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.297+0000 7fea75553700 1 --2- 172.21.15.131:0/422280364 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fea701008c0 0x7fea70101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:09.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.297+0000 7fea75553700 1 --2- 172.21.15.131:0/422280364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea70101270 0x7fea70101d80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:09.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.297+0000 7fea75553700 1 -- 172.21.15.131:0/422280364 >> 172.21.15.131:0/422280364 conn(0x7fea70075f80 msgr2=0x7fea700fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:51:09.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.297+0000 7fea75553700 1 -- 172.21.15.131:0/422280364 shutdown_connections 2024-02-05T07:51:09.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:09.297+0000 7fea75553700 1 -- 172.21.15.131:0/422280364 wait complete. 2024-02-05T07:51:09.302 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-02-05T07:51:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:09 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:10.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:09 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:09 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:10 smithi131 ceph-mon[29507]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:10 smithi131 ceph-mon[29507]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:51:11.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:10 smithi131 ceph-mon[41654]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:11.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:10 smithi131 ceph-mon[41654]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:51:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:10 smithi157 ceph-mon[33345]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:10 smithi157 ceph-mon[33345]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:51:11.222 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-02-05T07:51:09.272660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596316,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1008,"num_read_kb":854,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1771214,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":240308,"kb_used_data":7516,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":749720908,"statfs":{"total":767960285184,"available":767714209792,"internally_reserved":0,"allocated":7696384,"data_stored":4078876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002981"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439220+0000","last_change":"2024-02-05T07:49:53.478936+0000","last_active":"2024-02-05T07:50:27.439220+0000","last_peered":"2024-02-05T07:50:27.439220+0000","last_clean":"2024-02-05T07:50:27.439220+0000","last_became_active":"2024-02-05T07:49:53.478668+0000","last_became_peered":"2024-02-05T07:49:53.478668+0000","last_unstale":"2024-02-05T07:50:27.439220+0000","last_undegraded":"2024-02-05T07:50:27.439220+0000","last_fullsized":"2024-02-05T07:50:27.439220+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:36:17.460657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333104+0000","last_change":"2024-02-05T07:49:51.468784+0000","last_active":"2024-02-05T07:50:27.333104+0000","last_peered":"2024-02-05T07:50:27.333104+0000","last_clean":"2024-02-05T07:50:27.333104+0000","last_became_active":"2024-02-05T07:49:51.468458+0000","last_became_peered":"2024-02-05T07:49:51.468458+0000","last_unstale":"2024-02-05T07:50:27.333104+0000","last_undegraded":"2024-02-05T07:50:27.333104+0000","last_fullsized":"2024-02-05T07:50:27.333104+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:52:57.305576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256230+0000","last_change":"2024-02-05T07:49:47.519575+0000","last_active":"2024-02-05T07:50:27.256230+0000","last_peered":"2024-02-05T07:50:27.256230+0000","last_clean":"2024-02-05T07:50:27.256230+0000","last_became_active":"2024-02-05T07:49:47.519170+0000","last_became_peered":"2024-02-05T07:49:47.519170+0000","last_unstale":"2024-02-05T07:50:27.256230+0000","last_undegraded":"2024-02-05T07:50:27.256230+0000","last_fullsized":"2024-02-05T07:50:27.256230+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:22:01.546829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439146+0000","last_change":"2024-02-05T07:49:49.447163+0000","last_active":"2024-02-05T07:50:27.439146+0000","last_peered":"2024-02-05T07:50:27.439146+0000","last_clean":"2024-02-05T07:50:27.439146+0000","last_became_active":"2024-02-05T07:49:49.446839+0000","last_became_peered":"2024-02-05T07:49:49.446839+0000","last_unstale":"2024-02-05T07:50:27.439146+0000","last_undegraded":"2024-02-05T07:50:27.439146+0000","last_fullsized":"2024-02-05T07:50:27.439146+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:00:53.571872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253816+0000","last_change":"2024-02-05T07:49:53.835672+0000","last_active":"2024-02-05T07:50:27.253816+0000","last_peered":"2024-02-05T07:50:27.253816+0000","last_clean":"2024-02-05T07:50:27.253816+0000","last_became_active":"2024-02-05T07:49:53.835335+0000","last_became_peered":"2024-02-05T07:49:53.835335+0000","last_unstale":"2024-02-05T07:50:27.253816+0000","last_undegraded":"2024-02-05T07:50:27.253816+0000","last_fullsized":"2024-02-05T07:50:27.253816+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:40:15.864719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439145+0000","last_change":"2024-02-05T07:49:51.466888+0000","last_active":"2024-02-05T07:50:27.439145+0000","last_peered":"2024-02-05T07:50:27.439145+0000","last_clean":"2024-02-05T07:50:27.439145+0000","last_became_active":"2024-02-05T07:49:51.466570+0000","last_became_peered":"2024-02-05T07:49:51.466570+0000","last_unstale":"2024-02-05T07:50:27.439145+0000","last_undegraded":"2024-02-05T07:50:27.439145+0000","last_fullsized":"2024-02-05T07:50:27.439145+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:40:05.187075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254439+0000","last_change":"2024-02-05T07:49:47.519371+0000","last_active":"2024-02-05T07:50:27.254439+0000","last_peered":"2024-02-05T07:50:27.254439+0000","last_clean":"2024-02-05T07:50:27.254439+0000","last_became_active":"2024-02-05T07:49:47.519066+0000","last_became_peered":"2024-02-05T07:49:47.519066+0000","last_unstale":"2024-02-05T07:50:27.254439+0000","last_undegraded":"2024-02-05T07:50:27.254439+0000","last_fullsized":"2024-02-05T07:50:27.254439+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:55:02.883024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252762+0000","last_change":"2024-02-05T07:49:49.454673+0000","last_active":"2024-02-05T07:50:27.252762+0000","last_peered":"2024-02-05T07:50:27.252762+0000","last_clean":"2024-02-05T07:50:27.252762+0000","last_became_active":"2024-02-05T07:49:49.454396+0000","last_became_peered":"2024-02-05T07:49:49.454396+0000","last_unstale":"2024-02-05T07:50:27.252762+0000","last_undegraded":"2024-02-05T07:50:27.252762+0000","last_fullsized":"2024-02-05T07:50:27.252762+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:34:38.007722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254592+0000","last_change":"2024-02-05T07:49:53.475148+0000","last_active":"2024-02-05T07:50:27.254592+0000","last_peered":"2024-02-05T07:50:27.254592+0000","last_clean":"2024-02-05T07:50:27.254592+0000","last_became_active":"2024-02-05T07:49:53.474422+0000","last_became_peered":"2024-02-05T07:49:53.474422+0000","last_unstale":"2024-02-05T07:50:27.254592+0000","last_undegraded":"2024-02-05T07:50:27.254592+0000","last_fullsized":"2024-02-05T07:50:27.254592+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:25:25.277540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359664+0000","last_change":"2024-02-05T07:49:51.465588+0000","last_active":"2024-02-05T07:50:27.359664+0000","last_peered":"2024-02-05T07:50:27.359664+0000","last_clean":"2024-02-05T07:50:27.359664+0000","last_became_active":"2024-02-05T07:49:51.465313+0000","last_became_peered":"2024-02-05T07:49:51.465313+0000","last_unstale":"2024-02-05T07:50:27.359664+0000","last_undegraded":"2024-02-05T07:50:27.359664+0000","last_fullsized":"2024-02-05T07:50:27.359664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:21:44.285873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359953+0000","last_change":"2024-02-05T07:49:47.451422+0000","last_active":"2024-02-05T07:50:27.359953+0000","last_peered":"2024-02-05T07:50:27.359953+0000","last_clean":"2024-02-05T07:50:27.359953+0000","last_became_active":"2024-02-05T07:49:47.450995+0000","last_became_peered":"2024-02-05T07:49:47.450995+0000","last_unstale":"2024-02-05T07:50:27.359953+0000","last_undegraded":"2024-02-05T07:50:27.359953+0000","last_fullsized":"2024-02-05T07:50:27.359953+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:53:01.222021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439385+0000","last_change":"2024-02-05T07:49:49.452032+0000","last_active":"2024-02-05T07:50:27.439385+0000","last_peered":"2024-02-05T07:50:27.439385+0000","last_clean":"2024-02-05T07:50:27.439385+0000","last_became_active":"2024-02-05T07:49:49.451230+0000","last_became_peered":"2024-02-05T07:49:49.451230+0000","last_unstale":"2024-02-05T07:50:27.439385+0000","last_undegraded":"2024-02-05T07:50:27.439385+0000","last_fullsized":"2024-02-05T07:50:27.439385+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:23:51.569743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359371+0000","last_change":"2024-02-05T07:49:53.481868+0000","last_active":"2024-02-05T07:50:27.359371+0000","last_peered":"2024-02-05T07:50:27.359371+0000","last_clean":"2024-02-05T07:50:27.359371+0000","last_became_active":"2024-02-05T07:49:53.481591+0000","last_became_peered":"2024-02-05T07:49:53.481591+0000","last_unstale":"2024-02-05T07:50:27.359371+0000","last_undegraded":"2024-02-05T07:50:27.359371+0000","last_fullsized":"2024-02-05T07:50:27.359371+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:48:20.838716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368394+0000","last_change":"2024-02-05T07:49:51.467253+0000","last_active":"2024-02-05T07:50:27.368394+0000","last_peered":"2024-02-05T07:50:27.368394+0000","last_clean":"2024-02-05T07:50:27.368394+0000","last_became_active":"2024-02-05T07:49:51.466963+0000","last_became_peered":"2024-02-05T07:49:51.466963+0000","last_unstale":"2024-02-05T07:50:27.368394+0000","last_undegraded":"2024-02-05T07:50:27.368394+0000","last_fullsized":"2024-02-05T07:50:27.368394+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:09:56.514606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359832+0000","last_change":"2024-02-05T07:49:47.447964+0000","last_active":"2024-02-05T07:50:27.359832+0000","last_peered":"2024-02-05T07:50:27.359832+0000","last_clean":"2024-02-05T07:50:27.359832+0000","last_became_active":"2024-02-05T07:49:47.447666+0000","last_became_peered":"2024-02-05T07:49:47.447666+0000","last_unstale":"2024-02-05T07:50:27.359832+0000","last_undegraded":"2024-02-05T07:50:27.359832+0000","last_fullsized":"2024-02-05T07:50:27.359832+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:27:28.586296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"70'5","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439543+0000","last_change":"2024-02-05T07:49:49.451900+0000","last_active":"2024-02-05T07:50:27.439543+0000","last_peered":"2024-02-05T07:50:27.439543+0000","last_clean":"2024-02-05T07:50:27.439543+0000","last_became_active":"2024-02-05T07:49:49.451587+0000","last_became_peered":"2024-02-05T07:49:49.451587+0000","last_unstale":"2024-02-05T07:50:27.439543+0000","last_undegraded":"2024-02-05T07:50:27.439543+0000","last_fullsized":"2024-02-05T07:50:27.439543+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:10:57.020252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.440239+0000","last_change":"2024-02-05T07:49:53.837827+0000","last_active":"2024-02-05T07:50:27.440239+0000","last_peered":"2024-02-05T07:50:27.440239+0000","last_clean":"2024-02-05T07:50:27.440239+0000","last_became_active":"2024-02-05T07:49:53.836685+0000","last_became_peered":"2024-02-05T07:49:53.836685+0000","last_unstale":"2024-02-05T07:50:27.440239+0000","last_undegraded":"2024-02-05T07:50:27.440239+0000","last_fullsized":"2024-02-05T07:50:27.440239+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:04:43.878414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333553+0000","last_change":"2024-02-05T07:49:51.466739+0000","last_active":"2024-02-05T07:50:27.333553+0000","last_peered":"2024-02-05T07:50:27.333553+0000","last_clean":"2024-02-05T07:50:27.333553+0000","last_became_active":"2024-02-05T07:49:51.466415+0000","last_became_peered":"2024-02-05T07:49:51.466415+0000","last_unstale":"2024-02-05T07:50:27.333553+0000","last_undegraded":"2024-02-05T07:50:27.333553+0000","last_fullsized":"2024-02-05T07:50:27.333553+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:01:51.310604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256608+0000","last_change":"2024-02-05T07:49:47.445972+0000","last_active":"2024-02-05T07:50:27.256608+0000","last_peered":"2024-02-05T07:50:27.256608+0000","last_clean":"2024-02-05T07:50:27.256608+0000","last_became_active":"2024-02-05T07:49:47.445408+0000","last_became_peered":"2024-02-05T07:49:47.445408+0000","last_unstale":"2024-02-05T07:50:27.256608+0000","last_undegraded":"2024-02-05T07:50:27.256608+0000","last_fullsized":"2024-02-05T07:50:27.256608+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:31:53.687093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.364945+0000","last_change":"2024-02-05T07:49:49.452582+0000","last_active":"2024-02-05T07:50:27.364945+0000","last_peered":"2024-02-05T07:50:27.364945+0000","last_clean":"2024-02-05T07:50:27.364945+0000","last_became_active":"2024-02-05T07:49:49.452268+0000","last_became_peered":"2024-02-05T07:49:49.452268+0000","last_unstale":"2024-02-05T07:50:27.364945+0000","last_undegraded":"2024-02-05T07:50:27.364945+0000","last_fullsized":"2024-02-05T07:50:27.364945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:23:50.259812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254136+0000","last_change":"2024-02-05T07:49:53.836986+0000","last_active":"2024-02-05T07:50:27.254136+0000","last_peered":"2024-02-05T07:50:27.254136+0000","last_clean":"2024-02-05T07:50:27.254136+0000","last_became_active":"2024-02-05T07:49:53.836599+0000","last_became_peered":"2024-02-05T07:49:53.836599+0000","last_unstale":"2024-02-05T07:50:27.254136+0000","last_undegraded":"2024-02-05T07:50:27.254136+0000","last_fullsized":"2024-02-05T07:50:27.254136+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:08:33.779724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439855+0000","last_change":"2024-02-05T07:49:51.469272+0000","last_active":"2024-02-05T07:50:27.439855+0000","last_peered":"2024-02-05T07:50:27.439855+0000","last_clean":"2024-02-05T07:50:27.439855+0000","last_became_active":"2024-02-05T07:49:51.469006+0000","last_became_peered":"2024-02-05T07:49:51.469006+0000","last_unstale":"2024-02-05T07:50:27.439855+0000","last_undegraded":"2024-02-05T07:50:27.439855+0000","last_fullsized":"2024-02-05T07:50:27.439855+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:40:56.285764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439463+0000","last_change":"2024-02-05T07:49:47.520229+0000","last_active":"2024-02-05T07:50:27.439463+0000","last_peered":"2024-02-05T07:50:27.439463+0000","last_clean":"2024-02-05T07:50:27.439463+0000","last_became_active":"2024-02-05T07:49:47.519898+0000","last_became_peered":"2024-02-05T07:49:47.519898+0000","last_unstale":"2024-02-05T07:50:27.439463+0000","last_undegraded":"2024-02-05T07:50:27.439463+0000","last_fullsized":"2024-02-05T07:50:27.439463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:42:11.731508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"70'12","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254756+0000","last_change":"2024-02-05T07:49:49.454016+0000","last_active":"2024-02-05T07:50:27.254756+0000","last_peered":"2024-02-05T07:50:27.254756+0000","last_clean":"2024-02-05T07:50:27.254756+0000","last_became_active":"2024-02-05T07:49:49.453713+0000","last_became_peered":"2024-02-05T07:49:49.453713+0000","last_unstale":"2024-02-05T07:50:27.254756+0000","last_undegraded":"2024-02-05T07:50:27.254756+0000","last_fullsized":"2024-02-05T07:50:27.254756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:11:42.647053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"70'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368746+0000","last_change":"2024-02-05T07:49:53.480710+0000","last_active":"2024-02-05T07:50:27.368746+0000","last_peered":"2024-02-05T07:50:27.368746+0000","last_clean":"2024-02-05T07:50:27.368746+0000","last_became_active":"2024-02-05T07:49:53.480449+0000","last_became_peered":"2024-02-05T07:49:53.480449+0000","last_unstale":"2024-02-05T07:50:27.368746+0000","last_undegraded":"2024-02-05T07:50:27.368746+0000","last_fullsized":"2024-02-05T07:50:27.368746+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:19:24.609571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"70'8","reported_seq":53,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:51:02.574663+0000","last_change":"2024-02-05T07:49:51.463317+0000","last_active":"2024-02-05T07:51:02.574663+0000","last_peered":"2024-02-05T07:51:02.574663+0000","last_clean":"2024-02-05T07:51:02.574663+0000","last_became_active":"2024-02-05T07:49:51.463042+0000","last_became_peered":"2024-02-05T07:49:51.463042+0000","last_unstale":"2024-02-05T07:51:02.574663+0000","last_undegraded":"2024-02-05T07:51:02.574663+0000","last_fullsized":"2024-02-05T07:51:02.574663+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:54:26.640021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"63'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333165+0000","last_change":"2024-02-05T07:49:47.447165+0000","last_active":"2024-02-05T07:50:27.333165+0000","last_peered":"2024-02-05T07:50:27.333165+0000","last_clean":"2024-02-05T07:50:27.333165+0000","last_became_active":"2024-02-05T07:49:47.446862+0000","last_became_peered":"2024-02-05T07:49:47.446862+0000","last_unstale":"2024-02-05T07:50:27.333165+0000","last_undegraded":"2024-02-05T07:50:27.333165+0000","last_fullsized":"2024-02-05T07:50:27.333165+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:46:43.539023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256606+0000","last_change":"2024-02-05T07:49:49.735032+0000","last_active":"2024-02-05T07:50:27.256606+0000","last_peered":"2024-02-05T07:50:27.256606+0000","last_clean":"2024-02-05T07:50:27.256606+0000","last_became_active":"2024-02-05T07:49:49.734750+0000","last_became_peered":"2024-02-05T07:49:49.734750+0000","last_unstale":"2024-02-05T07:50:27.256606+0000","last_undegraded":"2024-02-05T07:50:27.256606+0000","last_fullsized":"2024-02-05T07:50:27.256606+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:53:41.822618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333354+0000","last_change":"2024-02-05T07:49:53.474114+0000","last_active":"2024-02-05T07:50:27.333354+0000","last_peered":"2024-02-05T07:50:27.333354+0000","last_clean":"2024-02-05T07:50:27.333354+0000","last_became_active":"2024-02-05T07:49:53.473808+0000","last_became_peered":"2024-02-05T07:49:53.473808+0000","last_unstale":"2024-02-05T07:50:27.333354+0000","last_undegraded":"2024-02-05T07:50:27.333354+0000","last_fullsized":"2024-02-05T07:50:27.333354+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:39:52.085855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256847+0000","last_change":"2024-02-05T07:49:51.462445+0000","last_active":"2024-02-05T07:50:27.256847+0000","last_peered":"2024-02-05T07:50:27.256847+0000","last_clean":"2024-02-05T07:50:27.256847+0000","last_became_active":"2024-02-05T07:49:51.461679+0000","last_became_peered":"2024-02-05T07:49:51.461679+0000","last_unstale":"2024-02-05T07:50:27.256847+0000","last_undegraded":"2024-02-05T07:50:27.256847+0000","last_fullsized":"2024-02-05T07:50:27.256847+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:40:46.347573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252165+0000","last_change":"2024-02-05T07:49:47.443560+0000","last_active":"2024-02-05T07:50:27.252165+0000","last_peered":"2024-02-05T07:50:27.252165+0000","last_clean":"2024-02-05T07:50:27.252165+0000","last_became_active":"2024-02-05T07:49:47.443309+0000","last_became_peered":"2024-02-05T07:49:47.443309+0000","last_unstale":"2024-02-05T07:50:27.252165+0000","last_undegraded":"2024-02-05T07:50:27.252165+0000","last_fullsized":"2024-02-05T07:50:27.252165+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:26:54.714704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254073+0000","last_change":"2024-02-05T07:49:49.446484+0000","last_active":"2024-02-05T07:50:27.254073+0000","last_peered":"2024-02-05T07:50:27.254073+0000","last_clean":"2024-02-05T07:50:27.254073+0000","last_became_active":"2024-02-05T07:49:49.446165+0000","last_became_peered":"2024-02-05T07:49:49.446165+0000","last_unstale":"2024-02-05T07:50:27.254073+0000","last_undegraded":"2024-02-05T07:50:27.254073+0000","last_fullsized":"2024-02-05T07:50:27.254073+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:38:51.811836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368391+0000","last_change":"2024-02-05T07:49:53.480539+0000","last_active":"2024-02-05T07:50:27.368391+0000","last_peered":"2024-02-05T07:50:27.368391+0000","last_clean":"2024-02-05T07:50:27.368391+0000","last_became_active":"2024-02-05T07:49:53.480241+0000","last_became_peered":"2024-02-05T07:49:53.480241+0000","last_unstale":"2024-02-05T07:50:27.368391+0000","last_undegraded":"2024-02-05T07:50:27.368391+0000","last_fullsized":"2024-02-05T07:50:27.368391+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:40:28.872638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253088+0000","last_change":"2024-02-05T07:49:51.460300+0000","last_active":"2024-02-05T07:50:27.253088+0000","last_peered":"2024-02-05T07:50:27.253088+0000","last_clean":"2024-02-05T07:50:27.253088+0000","last_became_active":"2024-02-05T07:49:51.460032+0000","last_became_peered":"2024-02-05T07:49:51.460032+0000","last_unstale":"2024-02-05T07:50:27.253088+0000","last_undegraded":"2024-02-05T07:50:27.253088+0000","last_fullsized":"2024-02-05T07:50:27.253088+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:31:52.284614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"63'1","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256122+0000","last_change":"2024-02-05T07:49:47.447541+0000","last_active":"2024-02-05T07:50:27.256122+0000","last_peered":"2024-02-05T07:50:27.256122+0000","last_clean":"2024-02-05T07:50:27.256122+0000","last_became_active":"2024-02-05T07:49:47.447220+0000","last_became_peered":"2024-02-05T07:49:47.447220+0000","last_unstale":"2024-02-05T07:50:27.256122+0000","last_undegraded":"2024-02-05T07:50:27.256122+0000","last_fullsized":"2024-02-05T07:50:27.256122+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:19:56.908993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"70'4","reported_seq":35,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252652+0000","last_change":"2024-02-05T07:49:49.455803+0000","last_active":"2024-02-05T07:50:27.252652+0000","last_peered":"2024-02-05T07:50:27.252652+0000","last_clean":"2024-02-05T07:50:27.252652+0000","last_became_active":"2024-02-05T07:49:49.455410+0000","last_became_peered":"2024-02-05T07:49:49.455410+0000","last_unstale":"2024-02-05T07:50:27.252652+0000","last_undegraded":"2024-02-05T07:50:27.252652+0000","last_fullsized":"2024-02-05T07:50:27.252652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:24:44.281857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253655+0000","last_change":"2024-02-05T07:49:53.835566+0000","last_active":"2024-02-05T07:50:27.253655+0000","last_peered":"2024-02-05T07:50:27.253655+0000","last_clean":"2024-02-05T07:50:27.253655+0000","last_became_active":"2024-02-05T07:49:53.835262+0000","last_became_peered":"2024-02-05T07:49:53.835262+0000","last_unstale":"2024-02-05T07:50:27.253655+0000","last_undegraded":"2024-02-05T07:50:27.253655+0000","last_fullsized":"2024-02-05T07:50:27.253655+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:09:54.372653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368695+0000","last_change":"2024-02-05T07:49:51.464304+0000","last_active":"2024-02-05T07:50:27.368695+0000","last_peered":"2024-02-05T07:50:27.368695+0000","last_clean":"2024-02-05T07:50:27.368695+0000","last_became_active":"2024-02-05T07:49:51.464012+0000","last_became_peered":"2024-02-05T07:49:51.464012+0000","last_unstale":"2024-02-05T07:50:27.368695+0000","last_undegraded":"2024-02-05T07:50:27.368695+0000","last_fullsized":"2024-02-05T07:50:27.368695+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:37:36.201341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.360033+0000","last_change":"2024-02-05T07:49:47.446196+0000","last_active":"2024-02-05T07:50:27.360033+0000","last_peered":"2024-02-05T07:50:27.360033+0000","last_clean":"2024-02-05T07:50:27.360033+0000","last_became_active":"2024-02-05T07:49:47.445869+0000","last_became_peered":"2024-02-05T07:49:47.445869+0000","last_unstale":"2024-02-05T07:50:27.360033+0000","last_undegraded":"2024-02-05T07:50:27.360033+0000","last_fullsized":"2024-02-05T07:50:27.360033+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:40:35.985397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252651+0000","last_change":"2024-02-05T07:49:49.734819+0000","last_active":"2024-02-05T07:50:27.252651+0000","last_peered":"2024-02-05T07:50:27.252651+0000","last_clean":"2024-02-05T07:50:27.252651+0000","last_became_active":"2024-02-05T07:49:49.734577+0000","last_became_peered":"2024-02-05T07:49:49.734577+0000","last_unstale":"2024-02-05T07:50:27.252651+0000","last_undegraded":"2024-02-05T07:50:27.252651+0000","last_fullsized":"2024-02-05T07:50:27.252651+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:33:18.467860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254283+0000","last_change":"2024-02-05T07:49:53.475769+0000","last_active":"2024-02-05T07:50:27.254283+0000","last_peered":"2024-02-05T07:50:27.254283+0000","last_clean":"2024-02-05T07:50:27.254283+0000","last_became_active":"2024-02-05T07:49:53.475473+0000","last_became_peered":"2024-02-05T07:49:53.475473+0000","last_unstale":"2024-02-05T07:50:27.254283+0000","last_undegraded":"2024-02-05T07:50:27.254283+0000","last_fullsized":"2024-02-05T07:50:27.254283+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:14:53.728317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253836+0000","last_change":"2024-02-05T07:49:51.466176+0000","last_active":"2024-02-05T07:50:27.253836+0000","last_peered":"2024-02-05T07:50:27.253836+0000","last_clean":"2024-02-05T07:50:27.253836+0000","last_became_active":"2024-02-05T07:49:51.464949+0000","last_became_peered":"2024-02-05T07:49:51.464949+0000","last_unstale":"2024-02-05T07:50:27.253836+0000","last_undegraded":"2024-02-05T07:50:27.253836+0000","last_fullsized":"2024-02-05T07:50:27.253836+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:22:05.181262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368862+0000","last_change":"2024-02-05T07:49:47.450946+0000","last_active":"2024-02-05T07:50:27.368862+0000","last_peered":"2024-02-05T07:50:27.368862+0000","last_clean":"2024-02-05T07:50:27.368862+0000","last_became_active":"2024-02-05T07:49:47.450675+0000","last_became_peered":"2024-02-05T07:49:47.450675+0000","last_unstale":"2024-02-05T07:50:27.368862+0000","last_undegraded":"2024-02-05T07:50:27.368862+0000","last_fullsized":"2024-02-05T07:50:27.368862+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:51:55.185524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333769+0000","last_change":"2024-02-05T07:49:49.452987+0000","last_active":"2024-02-05T07:50:27.333769+0000","last_peered":"2024-02-05T07:50:27.333769+0000","last_clean":"2024-02-05T07:50:27.333769+0000","last_became_active":"2024-02-05T07:49:49.452710+0000","last_became_peered":"2024-02-05T07:49:49.452710+0000","last_unstale":"2024-02-05T07:50:27.333769+0000","last_undegraded":"2024-02-05T07:50:27.333769+0000","last_fullsized":"2024-02-05T07:50:27.333769+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:30:34.871151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252277+0000","last_change":"2024-02-05T07:49:53.481105+0000","last_active":"2024-02-05T07:50:27.252277+0000","last_peered":"2024-02-05T07:50:27.252277+0000","last_clean":"2024-02-05T07:50:27.252277+0000","last_became_active":"2024-02-05T07:49:53.480568+0000","last_became_peered":"2024-02-05T07:49:53.480568+0000","last_unstale":"2024-02-05T07:50:27.252277+0000","last_undegraded":"2024-02-05T07:50:27.252277+0000","last_fullsized":"2024-02-05T07:50:27.252277+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:18:36.474182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333668+0000","last_change":"2024-02-05T07:49:51.469593+0000","last_active":"2024-02-05T07:50:27.333668+0000","last_peered":"2024-02-05T07:50:27.333668+0000","last_clean":"2024-02-05T07:50:27.333668+0000","last_became_active":"2024-02-05T07:49:51.469274+0000","last_became_peered":"2024-02-05T07:49:51.469274+0000","last_unstale":"2024-02-05T07:50:27.333668+0000","last_undegraded":"2024-02-05T07:50:27.333668+0000","last_fullsized":"2024-02-05T07:50:27.333668+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:58:09.801021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.440045+0000","last_change":"2024-02-05T07:49:47.453529+0000","last_active":"2024-02-05T07:50:27.440045+0000","last_peered":"2024-02-05T07:50:27.440045+0000","last_clean":"2024-02-05T07:50:27.440045+0000","last_became_active":"2024-02-05T07:49:47.453167+0000","last_became_peered":"2024-02-05T07:49:47.453167+0000","last_unstale":"2024-02-05T07:50:27.440045+0000","last_undegraded":"2024-02-05T07:50:27.440045+0000","last_fullsized":"2024-02-05T07:50:27.440045+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:18:30.336500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439702+0000","last_change":"2024-02-05T07:49:49.445760+0000","last_active":"2024-02-05T07:50:27.439702+0000","last_peered":"2024-02-05T07:50:27.439702+0000","last_clean":"2024-02-05T07:50:27.439702+0000","last_became_active":"2024-02-05T07:49:49.445287+0000","last_became_peered":"2024-02-05T07:49:49.445287+0000","last_unstale":"2024-02-05T07:50:27.439702+0000","last_undegraded":"2024-02-05T07:50:27.439702+0000","last_fullsized":"2024-02-05T07:50:27.439702+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:33:06.759850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256289+0000","last_change":"2024-02-05T07:49:53.481538+0000","last_active":"2024-02-05T07:50:27.256289+0000","last_peered":"2024-02-05T07:50:27.256289+0000","last_clean":"2024-02-05T07:50:27.256289+0000","last_became_active":"2024-02-05T07:49:53.481176+0000","last_became_peered":"2024-02-05T07:49:53.481176+0000","last_unstale":"2024-02-05T07:50:27.256289+0000","last_undegraded":"2024-02-05T07:50:27.256289+0000","last_fullsized":"2024-02-05T07:50:27.256289+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:16:32.895287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:51:02.574638+0000","last_change":"2024-02-05T07:49:51.469592+0000","last_active":"2024-02-05T07:51:02.574638+0000","last_peered":"2024-02-05T07:51:02.574638+0000","last_clean":"2024-02-05T07:51:02.574638+0000","last_became_active":"2024-02-05T07:49:51.469322+0000","last_became_peered":"2024-02-05T07:49:51.469322+0000","last_unstale":"2024-02-05T07:51:02.574638+0000","last_undegraded":"2024-02-05T07:51:02.574638+0000","last_fullsized":"2024-02-05T07:51:02.574638+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:49:35.499256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368461+0000","last_change":"2024-02-05T07:49:47.519907+0000","last_active":"2024-02-05T07:50:27.368461+0000","last_peered":"2024-02-05T07:50:27.368461+0000","last_clean":"2024-02-05T07:50:27.368461+0000","last_became_active":"2024-02-05T07:49:47.519608+0000","last_became_peered":"2024-02-05T07:49:47.519608+0000","last_unstale":"2024-02-05T07:50:27.368461+0000","last_undegraded":"2024-02-05T07:50:27.368461+0000","last_fullsized":"2024-02-05T07:50:27.368461+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:34:11.853473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"70'10","reported_seq":62,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252610+0000","last_change":"2024-02-05T07:50:11.872571+0000","last_active":"2024-02-05T07:50:27.252610+0000","last_peered":"2024-02-05T07:50:27.252610+0000","last_clean":"2024-02-05T07:50:27.252610+0000","last_became_active":"2024-02-05T07:50:11.861148+0000","last_became_peered":"2024-02-05T07:50:11.861148+0000","last_unstale":"2024-02-05T07:50:27.252610+0000","last_undegraded":"2024-02-05T07:50:27.252610+0000","last_fullsized":"2024-02-05T07:50:27.252610+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:42:58.286059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.440364+0000","last_change":"2024-02-05T07:49:53.481966+0000","last_active":"2024-02-05T07:50:27.440364+0000","last_peered":"2024-02-05T07:50:27.440364+0000","last_clean":"2024-02-05T07:50:27.440364+0000","last_became_active":"2024-02-05T07:49:53.481119+0000","last_became_peered":"2024-02-05T07:49:53.481119+0000","last_unstale":"2024-02-05T07:50:27.440364+0000","last_undegraded":"2024-02-05T07:50:27.440364+0000","last_fullsized":"2024-02-05T07:50:27.440364+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:26:52.284632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:51:02.574486+0000","last_change":"2024-02-05T07:49:51.469279+0000","last_active":"2024-02-05T07:51:02.574486+0000","last_peered":"2024-02-05T07:51:02.574486+0000","last_clean":"2024-02-05T07:51:02.574486+0000","last_became_active":"2024-02-05T07:49:51.468985+0000","last_became_peered":"2024-02-05T07:49:51.468985+0000","last_unstale":"2024-02-05T07:51:02.574486+0000","last_undegraded":"2024-02-05T07:51:02.574486+0000","last_fullsized":"2024-02-05T07:51:02.574486+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:23:51.463404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.255389+0000","last_change":"2024-02-05T07:49:47.438685+0000","last_active":"2024-02-05T07:50:27.255389+0000","last_peered":"2024-02-05T07:50:27.255389+0000","last_clean":"2024-02-05T07:50:27.255389+0000","last_became_active":"2024-02-05T07:49:47.438378+0000","last_became_peered":"2024-02-05T07:49:47.438378+0000","last_unstale":"2024-02-05T07:50:27.255389+0000","last_undegraded":"2024-02-05T07:50:27.255389+0000","last_fullsized":"2024-02-05T07:50:27.255389+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:07:59.061901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.360173+0000","last_change":"2024-02-05T07:49:49.735895+0000","last_active":"2024-02-05T07:50:27.360173+0000","last_peered":"2024-02-05T07:50:27.360173+0000","last_clean":"2024-02-05T07:50:27.360173+0000","last_became_active":"2024-02-05T07:49:49.735596+0000","last_became_peered":"2024-02-05T07:49:49.735596+0000","last_unstale":"2024-02-05T07:50:27.360173+0000","last_undegraded":"2024-02-05T07:50:27.360173+0000","last_fullsized":"2024-02-05T07:50:27.360173+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:40:42.036089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.365135+0000","last_change":"2024-02-05T07:49:53.485285+0000","last_active":"2024-02-05T07:50:27.365135+0000","last_peered":"2024-02-05T07:50:27.365135+0000","last_clean":"2024-02-05T07:50:27.365135+0000","last_became_active":"2024-02-05T07:49:53.482379+0000","last_became_peered":"2024-02-05T07:49:53.482379+0000","last_unstale":"2024-02-05T07:50:27.365135+0000","last_undegraded":"2024-02-05T07:50:27.365135+0000","last_fullsized":"2024-02-05T07:50:27.365135+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:45:51.848267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254565+0000","last_change":"2024-02-05T07:49:51.472049+0000","last_active":"2024-02-05T07:50:27.254565+0000","last_peered":"2024-02-05T07:50:27.254565+0000","last_clean":"2024-02-05T07:50:27.254565+0000","last_became_active":"2024-02-05T07:49:51.471747+0000","last_became_peered":"2024-02-05T07:49:51.471747+0000","last_unstale":"2024-02-05T07:50:27.254565+0000","last_undegraded":"2024-02-05T07:50:27.254565+0000","last_fullsized":"2024-02-05T07:50:27.254565+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:51:35.136254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368392+0000","last_change":"2024-02-05T07:49:47.450415+0000","last_active":"2024-02-05T07:50:27.368392+0000","last_peered":"2024-02-05T07:50:27.368392+0000","last_clean":"2024-02-05T07:50:27.368392+0000","last_became_active":"2024-02-05T07:49:47.448698+0000","last_became_peered":"2024-02-05T07:49:47.448698+0000","last_unstale":"2024-02-05T07:50:27.368392+0000","last_undegraded":"2024-02-05T07:50:27.368392+0000","last_fullsized":"2024-02-05T07:50:27.368392+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:09:40.857335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256728+0000","last_change":"2024-02-05T07:49:49.735988+0000","last_active":"2024-02-05T07:50:27.256728+0000","last_peered":"2024-02-05T07:50:27.256728+0000","last_clean":"2024-02-05T07:50:27.256728+0000","last_became_active":"2024-02-05T07:49:49.735650+0000","last_became_peered":"2024-02-05T07:49:49.735650+0000","last_unstale":"2024-02-05T07:50:27.256728+0000","last_undegraded":"2024-02-05T07:50:27.256728+0000","last_fullsized":"2024-02-05T07:50:27.256728+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:02:56.402985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368804+0000","last_change":"2024-02-05T07:49:53.836415+0000","last_active":"2024-02-05T07:50:27.368804+0000","last_peered":"2024-02-05T07:50:27.368804+0000","last_clean":"2024-02-05T07:50:27.368804+0000","last_became_active":"2024-02-05T07:49:53.836158+0000","last_became_peered":"2024-02-05T07:49:53.836158+0000","last_unstale":"2024-02-05T07:50:27.368804+0000","last_undegraded":"2024-02-05T07:50:27.368804+0000","last_fullsized":"2024-02-05T07:50:27.368804+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:46:25.206392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.360122+0000","last_change":"2024-02-05T07:49:51.468826+0000","last_active":"2024-02-05T07:50:27.360122+0000","last_peered":"2024-02-05T07:50:27.360122+0000","last_clean":"2024-02-05T07:50:27.360122+0000","last_became_active":"2024-02-05T07:49:51.468529+0000","last_became_peered":"2024-02-05T07:49:51.468529+0000","last_unstale":"2024-02-05T07:50:27.360122+0000","last_undegraded":"2024-02-05T07:50:27.360122+0000","last_fullsized":"2024-02-05T07:50:27.360122+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:10:49.229035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253031+0000","last_change":"2024-02-05T07:49:47.447847+0000","last_active":"2024-02-05T07:50:27.253031+0000","last_peered":"2024-02-05T07:50:27.253031+0000","last_clean":"2024-02-05T07:50:27.253031+0000","last_became_active":"2024-02-05T07:49:47.447557+0000","last_became_peered":"2024-02-05T07:49:47.447557+0000","last_unstale":"2024-02-05T07:50:27.253031+0000","last_undegraded":"2024-02-05T07:50:27.253031+0000","last_fullsized":"2024-02-05T07:50:27.253031+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:39:14.469684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"70'6","reported_seq":38,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254451+0000","last_change":"2024-02-05T07:49:49.735684+0000","last_active":"2024-02-05T07:50:27.254451+0000","last_peered":"2024-02-05T07:50:27.254451+0000","last_clean":"2024-02-05T07:50:27.254451+0000","last_became_active":"2024-02-05T07:49:49.735297+0000","last_became_peered":"2024-02-05T07:49:49.735297+0000","last_unstale":"2024-02-05T07:50:27.254451+0000","last_undegraded":"2024-02-05T07:50:27.254451+0000","last_fullsized":"2024-02-05T07:50:27.254451+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:55:48.643476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"70'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359843+0000","last_change":"2024-02-05T07:49:53.834941+0000","last_active":"2024-02-05T07:50:27.359843+0000","last_peered":"2024-02-05T07:50:27.359843+0000","last_clean":"2024-02-05T07:50:27.359843+0000","last_became_active":"2024-02-05T07:49:53.834635+0000","last_became_peered":"2024-02-05T07:49:53.834635+0000","last_unstale":"2024-02-05T07:50:27.359843+0000","last_undegraded":"2024-02-05T07:50:27.359843+0000","last_fullsized":"2024-02-05T07:50:27.359843+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:48:15.158738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"63'2","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.369115+0000","last_change":"2024-02-05T07:49:47.448888+0000","last_active":"2024-02-05T07:50:27.369115+0000","last_peered":"2024-02-05T07:50:27.369115+0000","last_clean":"2024-02-05T07:50:27.369115+0000","last_became_active":"2024-02-05T07:49:47.448550+0000","last_became_peered":"2024-02-05T07:49:47.448550+0000","last_unstale":"2024-02-05T07:50:27.369115+0000","last_undegraded":"2024-02-05T07:50:27.369115+0000","last_fullsized":"2024-02-05T07:50:27.369115+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:26:28.653523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359435+0000","last_change":"2024-02-05T07:49:49.737467+0000","last_active":"2024-02-05T07:50:27.359435+0000","last_peered":"2024-02-05T07:50:27.359435+0000","last_clean":"2024-02-05T07:50:27.359435+0000","last_became_active":"2024-02-05T07:49:49.737086+0000","last_became_peered":"2024-02-05T07:49:49.737086+0000","last_unstale":"2024-02-05T07:50:27.359435+0000","last_undegraded":"2024-02-05T07:50:27.359435+0000","last_fullsized":"2024-02-05T07:50:27.359435+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:31:52.911508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:51:02.574625+0000","last_change":"2024-02-05T07:49:51.464483+0000","last_active":"2024-02-05T07:51:02.574625+0000","last_peered":"2024-02-05T07:51:02.574625+0000","last_clean":"2024-02-05T07:51:02.574625+0000","last_became_active":"2024-02-05T07:49:51.464208+0000","last_became_peered":"2024-02-05T07:49:51.464208+0000","last_unstale":"2024-02-05T07:51:02.574625+0000","last_undegraded":"2024-02-05T07:51:02.574625+0000","last_fullsized":"2024-02-05T07:51:02.574625+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:41:49.063996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253747+0000","last_change":"2024-02-05T07:49:53.482133+0000","last_active":"2024-02-05T07:50:27.253747+0000","last_peered":"2024-02-05T07:50:27.253747+0000","last_clean":"2024-02-05T07:50:27.253747+0000","last_became_active":"2024-02-05T07:49:53.481520+0000","last_became_peered":"2024-02-05T07:49:53.481520+0000","last_unstale":"2024-02-05T07:50:27.253747+0000","last_undegraded":"2024-02-05T07:50:27.253747+0000","last_fullsized":"2024-02-05T07:50:27.253747+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:54:15.842757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.369197+0000","last_change":"2024-02-05T07:49:47.449999+0000","last_active":"2024-02-05T07:50:27.369197+0000","last_peered":"2024-02-05T07:50:27.369197+0000","last_clean":"2024-02-05T07:50:27.369197+0000","last_became_active":"2024-02-05T07:49:47.448351+0000","last_became_peered":"2024-02-05T07:49:47.448351+0000","last_unstale":"2024-02-05T07:50:27.369197+0000","last_undegraded":"2024-02-05T07:50:27.369197+0000","last_fullsized":"2024-02-05T07:50:27.369197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:03:49.578642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"70'12","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439438+0000","last_change":"2024-02-05T07:49:49.451505+0000","last_active":"2024-02-05T07:50:27.439438+0000","last_peered":"2024-02-05T07:50:27.439438+0000","last_clean":"2024-02-05T07:50:27.439438+0000","last_became_active":"2024-02-05T07:49:49.451159+0000","last_became_peered":"2024-02-05T07:49:49.451159+0000","last_unstale":"2024-02-05T07:50:27.439438+0000","last_undegraded":"2024-02-05T07:50:27.439438+0000","last_fullsized":"2024-02-05T07:50:27.439438+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:53:36.284342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.364494+0000","last_change":"2024-02-05T07:49:51.469984+0000","last_active":"2024-02-05T07:50:27.364494+0000","last_peered":"2024-02-05T07:50:27.364494+0000","last_clean":"2024-02-05T07:50:27.364494+0000","last_became_active":"2024-02-05T07:49:51.469660+0000","last_became_peered":"2024-02-05T07:49:51.469660+0000","last_unstale":"2024-02-05T07:50:27.364494+0000","last_undegraded":"2024-02-05T07:50:27.364494+0000","last_fullsized":"2024-02-05T07:50:27.364494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:17:57.565953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368197+0000","last_change":"2024-02-05T07:49:53.484776+0000","last_active":"2024-02-05T07:50:27.368197+0000","last_peered":"2024-02-05T07:50:27.368197+0000","last_clean":"2024-02-05T07:50:27.368197+0000","last_became_active":"2024-02-05T07:49:53.484428+0000","last_became_peered":"2024-02-05T07:49:53.484428+0000","last_unstale":"2024-02-05T07:50:27.368197+0000","last_undegraded":"2024-02-05T07:50:27.368197+0000","last_fullsized":"2024-02-05T07:50:27.368197+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:35:23.891153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.364829+0000","last_change":"2024-02-05T07:49:51.461096+0000","last_active":"2024-02-05T07:50:27.364829+0000","last_peered":"2024-02-05T07:50:27.364829+0000","last_clean":"2024-02-05T07:50:27.364829+0000","last_became_active":"2024-02-05T07:49:51.460803+0000","last_became_peered":"2024-02-05T07:49:51.460803+0000","last_unstale":"2024-02-05T07:50:27.364829+0000","last_undegraded":"2024-02-05T07:50:27.364829+0000","last_fullsized":"2024-02-05T07:50:27.364829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:09:55.228750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.369025+0000","last_change":"2024-02-05T07:49:47.451314+0000","last_active":"2024-02-05T07:50:27.369025+0000","last_peered":"2024-02-05T07:50:27.369025+0000","last_clean":"2024-02-05T07:50:27.369025+0000","last_became_active":"2024-02-05T07:49:47.450794+0000","last_became_peered":"2024-02-05T07:49:47.450794+0000","last_unstale":"2024-02-05T07:50:27.369025+0000","last_undegraded":"2024-02-05T07:50:27.369025+0000","last_fullsized":"2024-02-05T07:50:27.369025+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:32:59.395201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"70'19","reported_seq":60,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253374+0000","last_change":"2024-02-05T07:49:49.734559+0000","last_active":"2024-02-05T07:50:27.253374+0000","last_peered":"2024-02-05T07:50:27.253374+0000","last_clean":"2024-02-05T07:50:27.253374+0000","last_became_active":"2024-02-05T07:49:49.734204+0000","last_became_peered":"2024-02-05T07:49:49.734204+0000","last_unstale":"2024-02-05T07:50:27.253374+0000","last_undegraded":"2024-02-05T07:50:27.253374+0000","last_fullsized":"2024-02-05T07:50:27.253374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:47:19.559079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254920+0000","last_change":"2024-02-05T07:49:53.477215+0000","last_active":"2024-02-05T07:50:27.254920+0000","last_peered":"2024-02-05T07:50:27.254920+0000","last_clean":"2024-02-05T07:50:27.254920+0000","last_became_active":"2024-02-05T07:49:53.476921+0000","last_became_peered":"2024-02-05T07:49:53.476921+0000","last_unstale":"2024-02-05T07:50:27.254920+0000","last_undegraded":"2024-02-05T07:50:27.254920+0000","last_fullsized":"2024-02-05T07:50:27.254920+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:07:12.633466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.364782+0000","last_change":"2024-02-05T07:49:51.465329+0000","last_active":"2024-02-05T07:50:27.364782+0000","last_peered":"2024-02-05T07:50:27.364782+0000","last_clean":"2024-02-05T07:50:27.364782+0000","last_became_active":"2024-02-05T07:49:51.464968+0000","last_became_peered":"2024-02-05T07:49:51.464968+0000","last_unstale":"2024-02-05T07:50:27.364782+0000","last_undegraded":"2024-02-05T07:50:27.364782+0000","last_fullsized":"2024-02-05T07:50:27.364782+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:06:39.051173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359874+0000","last_change":"2024-02-05T07:49:47.450354+0000","last_active":"2024-02-05T07:50:27.359874+0000","last_peered":"2024-02-05T07:50:27.359874+0000","last_clean":"2024-02-05T07:50:27.359874+0000","last_became_active":"2024-02-05T07:49:47.450004+0000","last_became_peered":"2024-02-05T07:49:47.450004+0000","last_unstale":"2024-02-05T07:50:27.359874+0000","last_undegraded":"2024-02-05T07:50:27.359874+0000","last_fullsized":"2024-02-05T07:50:27.359874+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:06:36.695339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256489+0000","last_change":"2024-02-05T07:49:49.448567+0000","last_active":"2024-02-05T07:50:27.256489+0000","last_peered":"2024-02-05T07:50:27.256489+0000","last_clean":"2024-02-05T07:50:27.256489+0000","last_became_active":"2024-02-05T07:49:49.448168+0000","last_became_peered":"2024-02-05T07:49:49.448168+0000","last_unstale":"2024-02-05T07:50:27.256489+0000","last_undegraded":"2024-02-05T07:50:27.256489+0000","last_fullsized":"2024-02-05T07:50:27.256489+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:56:08.567259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254062+0000","last_change":"2024-02-05T07:49:53.484201+0000","last_active":"2024-02-05T07:50:27.254062+0000","last_peered":"2024-02-05T07:50:27.254062+0000","last_clean":"2024-02-05T07:50:27.254062+0000","last_became_active":"2024-02-05T07:49:53.483871+0000","last_became_peered":"2024-02-05T07:49:53.483871+0000","last_unstale":"2024-02-05T07:50:27.254062+0000","last_undegraded":"2024-02-05T07:50:27.254062+0000","last_fullsized":"2024-02-05T07:50:27.254062+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:22:57.963594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"66'2","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359527+0000","last_change":"2024-02-05T07:49:50.453091+0000","last_active":"2024-02-05T07:50:27.359527+0000","last_peered":"2024-02-05T07:50:27.359527+0000","last_clean":"2024-02-05T07:50:27.359527+0000","last_became_active":"2024-02-05T07:49:46.429198+0000","last_became_peered":"2024-02-05T07:49:46.429198+0000","last_unstale":"2024-02-05T07:50:27.359527+0000","last_undegraded":"2024-02-05T07:50:27.359527+0000","last_fullsized":"2024-02-05T07:50:27.359527+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:45.413168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:45.413168+0000","last_clean_scrub_stamp":"2024-02-05T07:49:45.413168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:45:11.102138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018745019999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439666+0000","last_change":"2024-02-05T07:49:47.447553+0000","last_active":"2024-02-05T07:50:27.439666+0000","last_peered":"2024-02-05T07:50:27.439666+0000","last_clean":"2024-02-05T07:50:27.439666+0000","last_became_active":"2024-02-05T07:49:47.447233+0000","last_became_peered":"2024-02-05T07:49:47.447233+0000","last_unstale":"2024-02-05T07:50:27.439666+0000","last_undegraded":"2024-02-05T07:50:27.439666+0000","last_fullsized":"2024-02-05T07:50:27.439666+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:15:12.905659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"70'28","reported_seq":94,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:51:02.574478+0000","last_change":"2024-02-05T07:49:49.452680+0000","last_active":"2024-02-05T07:51:02.574478+0000","last_peered":"2024-02-05T07:51:02.574478+0000","last_clean":"2024-02-05T07:51:02.574478+0000","last_became_active":"2024-02-05T07:49:49.452238+0000","last_became_peered":"2024-02-05T07:49:49.452238+0000","last_unstale":"2024-02-05T07:51:02.574478+0000","last_undegraded":"2024-02-05T07:51:02.574478+0000","last_fullsized":"2024-02-05T07:51:02.574478+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:45:32.499108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.255778+0000","last_change":"2024-02-05T07:49:51.464098+0000","last_active":"2024-02-05T07:50:27.255778+0000","last_peered":"2024-02-05T07:50:27.255778+0000","last_clean":"2024-02-05T07:50:27.255778+0000","last_became_active":"2024-02-05T07:49:51.463787+0000","last_became_peered":"2024-02-05T07:49:51.463787+0000","last_unstale":"2024-02-05T07:50:27.255778+0000","last_undegraded":"2024-02-05T07:50:27.255778+0000","last_fullsized":"2024-02-05T07:50:27.255778+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:47:29.020193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.369226+0000","last_change":"2024-02-05T07:49:53.836380+0000","last_active":"2024-02-05T07:50:27.369226+0000","last_peered":"2024-02-05T07:50:27.369226+0000","last_clean":"2024-02-05T07:50:27.369226+0000","last_became_active":"2024-02-05T07:49:53.835312+0000","last_became_peered":"2024-02-05T07:49:53.835312+0000","last_unstale":"2024-02-05T07:50:27.369226+0000","last_undegraded":"2024-02-05T07:50:27.369226+0000","last_fullsized":"2024-02-05T07:50:27.369226+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:10:50.589863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333179+0000","last_change":"2024-02-05T07:49:47.519909+0000","last_active":"2024-02-05T07:50:27.333179+0000","last_peered":"2024-02-05T07:50:27.333179+0000","last_clean":"2024-02-05T07:50:27.333179+0000","last_became_active":"2024-02-05T07:49:47.519351+0000","last_became_peered":"2024-02-05T07:49:47.519351+0000","last_unstale":"2024-02-05T07:50:27.333179+0000","last_undegraded":"2024-02-05T07:50:27.333179+0000","last_fullsized":"2024-02-05T07:50:27.333179+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:14:28.389456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.364569+0000","last_change":"2024-02-05T07:49:50.449980+0000","last_active":"2024-02-05T07:50:27.364569+0000","last_peered":"2024-02-05T07:50:27.364569+0000","last_clean":"2024-02-05T07:50:27.364569+0000","last_became_active":"2024-02-05T07:49:46.427509+0000","last_became_peered":"2024-02-05T07:49:46.427509+0000","last_unstale":"2024-02-05T07:50:27.364569+0000","last_undegraded":"2024-02-05T07:50:27.364569+0000","last_fullsized":"2024-02-05T07:50:27.364569+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:45.413168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:45.413168+0000","last_clean_scrub_stamp":"2024-02-05T07:49:45.413168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:39:57.099803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042346799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"70'13","reported_seq":56,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333984+0000","last_change":"2024-02-05T07:49:49.450289+0000","last_active":"2024-02-05T07:50:27.333984+0000","last_peered":"2024-02-05T07:50:27.333984+0000","last_clean":"2024-02-05T07:50:27.333984+0000","last_became_active":"2024-02-05T07:49:49.449991+0000","last_became_peered":"2024-02-05T07:49:49.449991+0000","last_unstale":"2024-02-05T07:50:27.333984+0000","last_undegraded":"2024-02-05T07:50:27.333984+0000","last_fullsized":"2024-02-05T07:50:27.333984+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:45:17.974373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.365256+0000","last_change":"2024-02-05T07:49:51.460350+0000","last_active":"2024-02-05T07:50:27.365256+0000","last_peered":"2024-02-05T07:50:27.365256+0000","last_clean":"2024-02-05T07:50:27.365256+0000","last_became_active":"2024-02-05T07:49:51.460020+0000","last_became_peered":"2024-02-05T07:49:51.460020+0000","last_unstale":"2024-02-05T07:50:27.365256+0000","last_undegraded":"2024-02-05T07:50:27.365256+0000","last_fullsized":"2024-02-05T07:50:27.365256+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:58:49.326928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.360179+0000","last_change":"2024-02-05T07:49:53.477271+0000","last_active":"2024-02-05T07:50:27.360179+0000","last_peered":"2024-02-05T07:50:27.360179+0000","last_clean":"2024-02-05T07:50:27.360179+0000","last_became_active":"2024-02-05T07:49:53.476972+0000","last_became_peered":"2024-02-05T07:49:53.476972+0000","last_unstale":"2024-02-05T07:50:27.360179+0000","last_undegraded":"2024-02-05T07:50:27.360179+0000","last_fullsized":"2024-02-05T07:50:27.360179+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:44:05.087061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"74'128","reported_seq":279,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:29.348548+0000","last_change":"2024-02-05T07:49:37.739906+0000","last_active":"2024-02-05T07:50:29.348548+0000","last_peered":"2024-02-05T07:50:29.348548+0000","last_clean":"2024-02-05T07:50:29.348548+0000","last_became_active":"2024-02-05T07:49:37.739474+0000","last_became_peered":"2024-02-05T07:49:37.739474+0000","last_unstale":"2024-02-05T07:50:29.348548+0000","last_undegraded":"2024-02-05T07:50:29.348548+0000","last_fullsized":"2024-02-05T07:50:29.348548+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:47:29.019622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:47:29.019622+0000","last_clean_scrub_stamp":"2024-02-05T07:47:29.019622+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:05:41.765759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253652+0000","last_change":"2024-02-05T07:49:47.446478+0000","last_active":"2024-02-05T07:50:27.253652+0000","last_peered":"2024-02-05T07:50:27.253652+0000","last_clean":"2024-02-05T07:50:27.253652+0000","last_became_active":"2024-02-05T07:49:47.446211+0000","last_became_peered":"2024-02-05T07:49:47.446211+0000","last_unstale":"2024-02-05T07:50:27.253652+0000","last_undegraded":"2024-02-05T07:50:27.253652+0000","last_fullsized":"2024-02-05T07:50:27.253652+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:48:08.108256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"70'16","reported_seq":68,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:51:02.574682+0000","last_change":"2024-02-05T07:49:49.735901+0000","last_active":"2024-02-05T07:51:02.574682+0000","last_peered":"2024-02-05T07:51:02.574682+0000","last_clean":"2024-02-05T07:51:02.574682+0000","last_became_active":"2024-02-05T07:49:49.735541+0000","last_became_peered":"2024-02-05T07:49:49.735541+0000","last_unstale":"2024-02-05T07:51:02.574682+0000","last_undegraded":"2024-02-05T07:51:02.574682+0000","last_fullsized":"2024-02-05T07:51:02.574682+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:45:11.927584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368967+0000","last_change":"2024-02-05T07:49:51.462683+0000","last_active":"2024-02-05T07:50:27.368967+0000","last_peered":"2024-02-05T07:50:27.368967+0000","last_clean":"2024-02-05T07:50:27.368967+0000","last_became_active":"2024-02-05T07:49:51.462419+0000","last_became_peered":"2024-02-05T07:49:51.462419+0000","last_unstale":"2024-02-05T07:50:27.368967+0000","last_undegraded":"2024-02-05T07:50:27.368967+0000","last_fullsized":"2024-02-05T07:50:27.368967+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:10:54.226467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333275+0000","last_change":"2024-02-05T07:49:53.481952+0000","last_active":"2024-02-05T07:50:27.333275+0000","last_peered":"2024-02-05T07:50:27.333275+0000","last_clean":"2024-02-05T07:50:27.333275+0000","last_became_active":"2024-02-05T07:49:53.481621+0000","last_became_peered":"2024-02-05T07:49:53.481621+0000","last_unstale":"2024-02-05T07:50:27.333275+0000","last_undegraded":"2024-02-05T07:50:27.333275+0000","last_fullsized":"2024-02-05T07:50:27.333275+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:40:05.915673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.255115+0000","last_change":"2024-02-05T07:49:47.449219+0000","last_active":"2024-02-05T07:50:27.255115+0000","last_peered":"2024-02-05T07:50:27.255115+0000","last_clean":"2024-02-05T07:50:27.255115+0000","last_became_active":"2024-02-05T07:49:47.448931+0000","last_became_peered":"2024-02-05T07:49:47.448931+0000","last_unstale":"2024-02-05T07:50:27.255115+0000","last_undegraded":"2024-02-05T07:50:27.255115+0000","last_fullsized":"2024-02-05T07:50:27.255115+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:20:04.989220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"71'5","reported_seq":119,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:51:07.229205+0000","last_change":"2024-02-05T07:49:50.456166+0000","last_active":"2024-02-05T07:51:07.229205+0000","last_peered":"2024-02-05T07:51:07.229205+0000","last_clean":"2024-02-05T07:51:07.229205+0000","last_became_active":"2024-02-05T07:49:46.433572+0000","last_became_peered":"2024-02-05T07:49:46.433572+0000","last_unstale":"2024-02-05T07:51:07.229205+0000","last_undegraded":"2024-02-05T07:51:07.229205+0000","last_fullsized":"2024-02-05T07:51:07.229205+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:45.413168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:45.413168+0000","last_clean_scrub_stamp":"2024-02-05T07:49:45.413168+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:51:34.023397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015152,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":74,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"70'12","reported_seq":47,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256393+0000","last_change":"2024-02-05T07:49:49.447760+0000","last_active":"2024-02-05T07:50:27.256393+0000","last_peered":"2024-02-05T07:50:27.256393+0000","last_clean":"2024-02-05T07:50:27.256393+0000","last_became_active":"2024-02-05T07:49:49.447432+0000","last_became_peered":"2024-02-05T07:49:49.447432+0000","last_unstale":"2024-02-05T07:50:27.256393+0000","last_undegraded":"2024-02-05T07:50:27.256393+0000","last_fullsized":"2024-02-05T07:50:27.256393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:56:25.889695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.360392+0000","last_change":"2024-02-05T07:49:51.468986+0000","last_active":"2024-02-05T07:50:27.360392+0000","last_peered":"2024-02-05T07:50:27.360392+0000","last_clean":"2024-02-05T07:50:27.360392+0000","last_became_active":"2024-02-05T07:49:51.468720+0000","last_became_peered":"2024-02-05T07:49:51.468720+0000","last_unstale":"2024-02-05T07:50:27.360392+0000","last_undegraded":"2024-02-05T07:50:27.360392+0000","last_fullsized":"2024-02-05T07:50:27.360392+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:11:27.089767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333044+0000","last_change":"2024-02-05T07:49:53.835527+0000","last_active":"2024-02-05T07:50:27.333044+0000","last_peered":"2024-02-05T07:50:27.333044+0000","last_clean":"2024-02-05T07:50:27.333044+0000","last_became_active":"2024-02-05T07:49:53.834607+0000","last_became_peered":"2024-02-05T07:49:53.834607+0000","last_unstale":"2024-02-05T07:50:27.333044+0000","last_undegraded":"2024-02-05T07:50:27.333044+0000","last_fullsized":"2024-02-05T07:50:27.333044+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:57:38.229214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"63'1","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333727+0000","last_change":"2024-02-05T07:49:47.519808+0000","last_active":"2024-02-05T07:50:27.333727+0000","last_peered":"2024-02-05T07:50:27.333727+0000","last_clean":"2024-02-05T07:50:27.333727+0000","last_became_active":"2024-02-05T07:49:47.519528+0000","last_became_peered":"2024-02-05T07:49:47.519528+0000","last_unstale":"2024-02-05T07:50:27.333727+0000","last_undegraded":"2024-02-05T07:50:27.333727+0000","last_fullsized":"2024-02-05T07:50:27.333727+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:19:29.206520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"70'19","reported_seq":65,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256663+0000","last_change":"2024-02-05T07:49:49.734608+0000","last_active":"2024-02-05T07:50:27.256663+0000","last_peered":"2024-02-05T07:50:27.256663+0000","last_clean":"2024-02-05T07:50:27.256663+0000","last_became_active":"2024-02-05T07:49:49.734296+0000","last_became_peered":"2024-02-05T07:49:49.734296+0000","last_unstale":"2024-02-05T07:50:27.256663+0000","last_undegraded":"2024-02-05T07:50:27.256663+0000","last_fullsized":"2024-02-05T07:50:27.256663+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:16:40.799503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253379+0000","last_change":"2024-02-05T07:49:51.464033+0000","last_active":"2024-02-05T07:50:27.253379+0000","last_peered":"2024-02-05T07:50:27.253379+0000","last_clean":"2024-02-05T07:50:27.253379+0000","last_became_active":"2024-02-05T07:49:51.463720+0000","last_became_peered":"2024-02-05T07:49:51.463720+0000","last_unstale":"2024-02-05T07:50:27.253379+0000","last_undegraded":"2024-02-05T07:50:27.253379+0000","last_fullsized":"2024-02-05T07:50:27.253379+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:45:44.086697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254382+0000","last_change":"2024-02-05T07:49:53.483687+0000","last_active":"2024-02-05T07:50:27.254382+0000","last_peered":"2024-02-05T07:50:27.254382+0000","last_clean":"2024-02-05T07:50:27.254382+0000","last_became_active":"2024-02-05T07:49:53.483407+0000","last_became_peered":"2024-02-05T07:49:53.483407+0000","last_unstale":"2024-02-05T07:50:27.254382+0000","last_undegraded":"2024-02-05T07:50:27.254382+0000","last_fullsized":"2024-02-05T07:50:27.254382+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:00:31.813086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.360071+0000","last_change":"2024-02-05T07:49:47.519813+0000","last_active":"2024-02-05T07:50:27.360071+0000","last_peered":"2024-02-05T07:50:27.360071+0000","last_clean":"2024-02-05T07:50:27.360071+0000","last_became_active":"2024-02-05T07:49:47.519527+0000","last_became_peered":"2024-02-05T07:49:47.519527+0000","last_unstale":"2024-02-05T07:50:27.360071+0000","last_undegraded":"2024-02-05T07:50:27.360071+0000","last_fullsized":"2024-02-05T07:50:27.360071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:54:09.621261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"70'10","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.364466+0000","last_change":"2024-02-05T07:50:11.867536+0000","last_active":"2024-02-05T07:50:27.364466+0000","last_peered":"2024-02-05T07:50:27.364466+0000","last_clean":"2024-02-05T07:50:27.364466+0000","last_became_active":"2024-02-05T07:50:11.859627+0000","last_became_peered":"2024-02-05T07:50:11.859627+0000","last_unstale":"2024-02-05T07:50:27.364466+0000","last_undegraded":"2024-02-05T07:50:27.364466+0000","last_fullsized":"2024-02-05T07:50:27.364466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:22:14.912788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:51:02.574434+0000","last_change":"2024-02-05T07:49:51.462401+0000","last_active":"2024-02-05T07:51:02.574434+0000","last_peered":"2024-02-05T07:51:02.574434+0000","last_clean":"2024-02-05T07:51:02.574434+0000","last_became_active":"2024-02-05T07:49:51.461018+0000","last_became_peered":"2024-02-05T07:49:51.461018+0000","last_unstale":"2024-02-05T07:51:02.574434+0000","last_undegraded":"2024-02-05T07:51:02.574434+0000","last_fullsized":"2024-02-05T07:51:02.574434+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:25:38.200242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.368836+0000","last_change":"2024-02-05T07:49:53.836234+0000","last_active":"2024-02-05T07:50:27.368836+0000","last_peered":"2024-02-05T07:50:27.368836+0000","last_clean":"2024-02-05T07:50:27.368836+0000","last_became_active":"2024-02-05T07:49:53.835945+0000","last_became_peered":"2024-02-05T07:49:53.835945+0000","last_unstale":"2024-02-05T07:50:27.368836+0000","last_undegraded":"2024-02-05T07:50:27.368836+0000","last_fullsized":"2024-02-05T07:50:27.368836+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:05:54.982258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"63'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.256514+0000","last_change":"2024-02-05T07:49:47.445692+0000","last_active":"2024-02-05T07:50:27.256514+0000","last_peered":"2024-02-05T07:50:27.256514+0000","last_clean":"2024-02-05T07:50:27.256514+0000","last_became_active":"2024-02-05T07:49:47.445365+0000","last_became_peered":"2024-02-05T07:49:47.445365+0000","last_unstale":"2024-02-05T07:50:27.256514+0000","last_undegraded":"2024-02-05T07:50:27.256514+0000","last_fullsized":"2024-02-05T07:50:27.256514+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:05:47.500048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"70'14","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439367+0000","last_change":"2024-02-05T07:49:49.442410+0000","last_active":"2024-02-05T07:50:27.439367+0000","last_peered":"2024-02-05T07:50:27.439367+0000","last_clean":"2024-02-05T07:50:27.439367+0000","last_became_active":"2024-02-05T07:49:49.441931+0000","last_became_peered":"2024-02-05T07:49:49.441931+0000","last_unstale":"2024-02-05T07:50:27.439367+0000","last_undegraded":"2024-02-05T07:50:27.439367+0000","last_fullsized":"2024-02-05T07:50:27.439367+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:59:10.646362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"70'8","reported_seq":53,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:51:02.574515+0000","last_change":"2024-02-05T07:49:51.467071+0000","last_active":"2024-02-05T07:51:02.574515+0000","last_peered":"2024-02-05T07:51:02.574515+0000","last_clean":"2024-02-05T07:51:02.574515+0000","last_became_active":"2024-02-05T07:49:51.466715+0000","last_became_peered":"2024-02-05T07:49:51.466715+0000","last_unstale":"2024-02-05T07:51:02.574515+0000","last_undegraded":"2024-02-05T07:51:02.574515+0000","last_fullsized":"2024-02-05T07:51:02.574515+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:32:35.577834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439272+0000","last_change":"2024-02-05T07:49:53.479631+0000","last_active":"2024-02-05T07:50:27.439272+0000","last_peered":"2024-02-05T07:50:27.439272+0000","last_clean":"2024-02-05T07:50:27.439272+0000","last_became_active":"2024-02-05T07:49:53.479347+0000","last_became_peered":"2024-02-05T07:49:53.479347+0000","last_unstale":"2024-02-05T07:50:27.439272+0000","last_undegraded":"2024-02-05T07:50:27.439272+0000","last_fullsized":"2024-02-05T07:50:27.439272+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:42:12.523750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.254293+0000","last_change":"2024-02-05T07:49:47.440757+0000","last_active":"2024-02-05T07:50:27.254293+0000","last_peered":"2024-02-05T07:50:27.254293+0000","last_clean":"2024-02-05T07:50:27.254293+0000","last_became_active":"2024-02-05T07:49:47.440474+0000","last_became_peered":"2024-02-05T07:49:47.440474+0000","last_unstale":"2024-02-05T07:50:27.254293+0000","last_undegraded":"2024-02-05T07:50:27.254293+0000","last_fullsized":"2024-02-05T07:50:27.254293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:54:48.519955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"70'18","reported_seq":61,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252421+0000","last_change":"2024-02-05T07:49:49.736372+0000","last_active":"2024-02-05T07:50:27.252421+0000","last_peered":"2024-02-05T07:50:27.252421+0000","last_clean":"2024-02-05T07:50:27.252421+0000","last_became_active":"2024-02-05T07:49:49.735956+0000","last_became_peered":"2024-02-05T07:49:49.735956+0000","last_unstale":"2024-02-05T07:50:27.252421+0000","last_undegraded":"2024-02-05T07:50:27.252421+0000","last_fullsized":"2024-02-05T07:50:27.252421+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:42:41.997513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439194+0000","last_change":"2024-02-05T07:49:51.465286+0000","last_active":"2024-02-05T07:50:27.439194+0000","last_peered":"2024-02-05T07:50:27.439194+0000","last_clean":"2024-02-05T07:50:27.439194+0000","last_became_active":"2024-02-05T07:49:51.464861+0000","last_became_peered":"2024-02-05T07:49:51.464861+0000","last_unstale":"2024-02-05T07:50:27.439194+0000","last_undegraded":"2024-02-05T07:50:27.439194+0000","last_fullsized":"2024-02-05T07:50:27.439194+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:50:08.659066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.359774+0000","last_change":"2024-02-05T07:49:53.476564+0000","last_active":"2024-02-05T07:50:27.359774+0000","last_peered":"2024-02-05T07:50:27.359774+0000","last_clean":"2024-02-05T07:50:27.359774+0000","last_became_active":"2024-02-05T07:49:53.476286+0000","last_became_peered":"2024-02-05T07:49:53.476286+0000","last_unstale":"2024-02-05T07:50:27.359774+0000","last_undegraded":"2024-02-05T07:50:27.359774+0000","last_fullsized":"2024-02-05T07:50:27.359774+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:48:34.681644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:51:02.574623+0000","last_change":"2024-02-05T07:49:51.462755+0000","last_active":"2024-02-05T07:51:02.574623+0000","last_peered":"2024-02-05T07:51:02.574623+0000","last_clean":"2024-02-05T07:51:02.574623+0000","last_became_active":"2024-02-05T07:49:51.462362+0000","last_became_peered":"2024-02-05T07:49:51.462362+0000","last_unstale":"2024-02-05T07:51:02.574623+0000","last_undegraded":"2024-02-05T07:51:02.574623+0000","last_fullsized":"2024-02-05T07:51:02.574623+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:36:47.969011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252336+0000","last_change":"2024-02-05T07:49:47.443430+0000","last_active":"2024-02-05T07:50:27.252336+0000","last_peered":"2024-02-05T07:50:27.252336+0000","last_clean":"2024-02-05T07:50:27.252336+0000","last_became_active":"2024-02-05T07:49:47.443157+0000","last_became_peered":"2024-02-05T07:49:47.443157+0000","last_unstale":"2024-02-05T07:50:27.252336+0000","last_undegraded":"2024-02-05T07:50:27.252336+0000","last_fullsized":"2024-02-05T07:50:27.252336+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:35:21.792177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333952+0000","last_change":"2024-02-05T07:49:49.452089+0000","last_active":"2024-02-05T07:50:27.333952+0000","last_peered":"2024-02-05T07:50:27.333952+0000","last_clean":"2024-02-05T07:50:27.333952+0000","last_became_active":"2024-02-05T07:49:49.451704+0000","last_became_peered":"2024-02-05T07:49:49.451704+0000","last_unstale":"2024-02-05T07:50:27.333952+0000","last_undegraded":"2024-02-05T07:50:27.333952+0000","last_fullsized":"2024-02-05T07:50:27.333952+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:35:06.578768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.252463+0000","last_change":"2024-02-05T07:49:53.835233+0000","last_active":"2024-02-05T07:50:27.252463+0000","last_peered":"2024-02-05T07:50:27.252463+0000","last_clean":"2024-02-05T07:50:27.252463+0000","last_became_active":"2024-02-05T07:49:53.834847+0000","last_became_peered":"2024-02-05T07:49:53.834847+0000","last_unstale":"2024-02-05T07:50:27.252463+0000","last_undegraded":"2024-02-05T07:50:27.252463+0000","last_fullsized":"2024-02-05T07:50:27.252463+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:17:06.871213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.360355+0000","last_change":"2024-02-05T07:49:51.466399+0000","last_active":"2024-02-05T07:50:27.360355+0000","last_peered":"2024-02-05T07:50:27.360355+0000","last_clean":"2024-02-05T07:50:27.360355+0000","last_became_active":"2024-02-05T07:49:51.466099+0000","last_became_peered":"2024-02-05T07:49:51.466099+0000","last_unstale":"2024-02-05T07:50:27.360355+0000","last_undegraded":"2024-02-05T07:50:27.360355+0000","last_fullsized":"2024-02-05T07:50:27.360355+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:43:11.141949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439260+0000","last_change":"2024-02-05T07:49:47.519778+0000","last_active":"2024-02-05T07:50:27.439260+0000","last_peered":"2024-02-05T07:50:27.439260+0000","last_clean":"2024-02-05T07:50:27.439260+0000","last_became_active":"2024-02-05T07:49:47.519469+0000","last_became_peered":"2024-02-05T07:49:47.519469+0000","last_unstale":"2024-02-05T07:50:27.439260+0000","last_undegraded":"2024-02-05T07:50:27.439260+0000","last_fullsized":"2024-02-05T07:50:27.439260+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:37:24.792568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"70'11","reported_seq":66,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.440225+0000","last_change":"2024-02-05T07:50:11.866569+0000","last_active":"2024-02-05T07:50:27.440225+0000","last_peered":"2024-02-05T07:50:27.440225+0000","last_clean":"2024-02-05T07:50:27.440225+0000","last_became_active":"2024-02-05T07:50:11.857881+0000","last_became_peered":"2024-02-05T07:50:11.857881+0000","last_unstale":"2024-02-05T07:50:27.440225+0000","last_undegraded":"2024-02-05T07:50:27.440225+0000","last_fullsized":"2024-02-05T07:50:27.440225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:27:32.387706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.365277+0000","last_change":"2024-02-05T07:49:53.485182+0000","last_active":"2024-02-05T07:50:27.365277+0000","last_peered":"2024-02-05T07:50:27.365277+0000","last_clean":"2024-02-05T07:50:27.365277+0000","last_became_active":"2024-02-05T07:49:53.484590+0000","last_became_peered":"2024-02-05T07:49:53.484590+0000","last_unstale":"2024-02-05T07:50:27.365277+0000","last_undegraded":"2024-02-05T07:50:27.365277+0000","last_fullsized":"2024-02-05T07:50:27.365277+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:19:06.287158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.333840+0000","last_change":"2024-02-05T07:49:51.468829+0000","last_active":"2024-02-05T07:50:27.333840+0000","last_peered":"2024-02-05T07:50:27.333840+0000","last_clean":"2024-02-05T07:50:27.333840+0000","last_became_active":"2024-02-05T07:49:51.468495+0000","last_became_peered":"2024-02-05T07:49:51.468495+0000","last_unstale":"2024-02-05T07:50:27.333840+0000","last_undegraded":"2024-02-05T07:50:27.333840+0000","last_fullsized":"2024-02-05T07:50:27.333840+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:50:32.323195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253271+0000","last_change":"2024-02-05T07:49:47.449403+0000","last_active":"2024-02-05T07:50:27.253271+0000","last_peered":"2024-02-05T07:50:27.253271+0000","last_clean":"2024-02-05T07:50:27.253271+0000","last_became_active":"2024-02-05T07:49:47.448970+0000","last_became_peered":"2024-02-05T07:49:47.448970+0000","last_unstale":"2024-02-05T07:50:27.253271+0000","last_undegraded":"2024-02-05T07:50:27.253271+0000","last_fullsized":"2024-02-05T07:50:27.253271+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:33:28.806519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"70'17","reported_seq":57,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.440139+0000","last_change":"2024-02-05T07:49:49.446195+0000","last_active":"2024-02-05T07:50:27.440139+0000","last_peered":"2024-02-05T07:50:27.440139+0000","last_clean":"2024-02-05T07:50:27.440139+0000","last_became_active":"2024-02-05T07:49:49.445922+0000","last_became_peered":"2024-02-05T07:49:49.445922+0000","last_unstale":"2024-02-05T07:50:27.440139+0000","last_undegraded":"2024-02-05T07:50:27.440139+0000","last_fullsized":"2024-02-05T07:50:27.440139+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:03:12.272146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.440379+0000","last_change":"2024-02-05T07:49:53.480356+0000","last_active":"2024-02-05T07:50:27.440379+0000","last_peered":"2024-02-05T07:50:27.440379+0000","last_clean":"2024-02-05T07:50:27.440379+0000","last_became_active":"2024-02-05T07:49:53.480089+0000","last_became_peered":"2024-02-05T07:49:53.480089+0000","last_unstale":"2024-02-05T07:50:27.440379+0000","last_undegraded":"2024-02-05T07:50:27.440379+0000","last_fullsized":"2024-02-05T07:50:27.440379+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:52.450601+0000","last_clean_scrub_stamp":"2024-02-05T07:49:52.450601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:17:16.231336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:51:02.574445+0000","last_change":"2024-02-05T07:49:51.461674+0000","last_active":"2024-02-05T07:51:02.574445+0000","last_peered":"2024-02-05T07:51:02.574445+0000","last_clean":"2024-02-05T07:51:02.574445+0000","last_became_active":"2024-02-05T07:49:51.461393+0000","last_became_peered":"2024-02-05T07:49:51.461393+0000","last_unstale":"2024-02-05T07:51:02.574445+0000","last_undegraded":"2024-02-05T07:51:02.574445+0000","last_fullsized":"2024-02-05T07:51:02.574445+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:50.443462+0000","last_clean_scrub_stamp":"2024-02-05T07:49:50.443462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:22:33.553443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.253933+0000","last_change":"2024-02-05T07:49:47.447353+0000","last_active":"2024-02-05T07:50:27.253933+0000","last_peered":"2024-02-05T07:50:27.253933+0000","last_clean":"2024-02-05T07:50:27.253933+0000","last_became_active":"2024-02-05T07:49:47.447006+0000","last_became_peered":"2024-02-05T07:49:47.447006+0000","last_unstale":"2024-02-05T07:50:27.253933+0000","last_undegraded":"2024-02-05T07:50:27.253933+0000","last_fullsized":"2024-02-05T07:50:27.253933+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:46.417827+0000","last_clean_scrub_stamp":"2024-02-05T07:49:46.417827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:11:03.724451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:50:27.439903+0000","last_change":"2024-02-05T07:49:49.451678+0000","last_active":"2024-02-05T07:50:27.439903+0000","last_peered":"2024-02-05T07:50:27.439903+0000","last_clean":"2024-02-05T07:50:27.439903+0000","last_became_active":"2024-02-05T07:49:49.451264+0000","last_became_peered":"2024-02-05T07:49:49.451264+0000","last_unstale":"2024-02-05T07:50:27.439903+0000","last_undegraded":"2024-02-05T07:50:27.439903+0000","last_fullsized":"2024-02-05T07:50:27.439903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:49:48.429654+0000","last_clean_scrub_stamp":"2024-02-05T07:49:48.429654+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:13:15.797062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":69,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":56,"seq":240518168597,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32476,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712676,"statfs":{"total":95995035648,"available":95961780224,"internally_reserved":0,"allocated":1327104,"data_stored":878713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.35499999999999998}]},{"osd":1,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.29399999999999998}]},{"osd":2,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.38500000000000001}]},{"osd":3,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.52800000000000002}]},{"osd":4,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.51300000000000001}]},{"osd":5,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60299999999999998}]},{"osd":6,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57099999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462938,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32456,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712696,"statfs":{"total":95995035648,"available":95961800704,"internally_reserved":0,"allocated":1310720,"data_stored":877675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.503},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.52700000000000002}]},{"osd":1,"last update":"Mon Feb 5 07:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51600000000000001}]},{"osd":2,"last update":"Mon Feb 5 07:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.60699999999999998}]},{"osd":3,"last update":"Mon Feb 5 07:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.59199999999999997}]},{"osd":4,"last update":"Mon Feb 5 07:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.41299999999999998}]},{"osd":5,"last update":"Mon Feb 5 07:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.48199999999999998}]},{"osd":7,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.434},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.62}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27780,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717372,"statfs":{"total":95995035648,"available":95966588928,"internally_reserved":0,"allocated":778240,"data_stored":289367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.34399999999999997,"15min":0.31900000000000001},"min":{"1min":0.23300000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.36599999999999999,"15min":0.33100000000000002},"min":{"1min":0.28599999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.30299999999999999}]},{"osd":2,"last update":"Mon Feb 5 07:50:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.40500000000000003},"min":{"1min":0.30599999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60999999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.41499999999999998,"15min":0.40300000000000002},"min":{"1min":0.28699999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.27800000000000002}]},{"osd":3,"last update":"Mon Feb 5 07:50:56 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59099999999999997,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.42599999999999999,"15min":0.436},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.56699999999999995,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.31900000000000001}]},{"osd":4,"last update":"Mon Feb 5 07:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57199999999999995,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67100000000000004,"5min":0.751,"15min":0.751},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.74199999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.57199999999999995}]},{"osd":5,"last update":"Mon Feb 5 07:50:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70899999999999996,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.501},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.71799999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.60099999999999998}]},{"osd":6,"last update":"Mon Feb 5 07:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.52600000000000002}]},{"osd":7,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.58699999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28324,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716828,"statfs":{"total":95995035648,"available":95966031872,"internally_reserved":0,"allocated":1339392,"data_stored":879343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 5 07:51:07 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.35499999999999998,"15min":0.33300000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.53300000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.377,"15min":0.34499999999999997},"min":{"1min":0.245,"5min":0.223,"15min":0.223},"max":{"1min":0.57399999999999995,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.376}]},{"osd":2,"last update":"Mon Feb 5 07:50:41 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40699999999999997,"15min":0.40600000000000003},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54600000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.39500000000000002,"15min":0.38500000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.54900000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.33200000000000002}]},{"osd":3,"last update":"Mon Feb 5 07:51:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.47499999999999998,"15min":0.49099999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.56799999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.46400000000000002,"15min":0.48099999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.58499999999999996,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.29099999999999998}]},{"osd":4,"last update":"Mon Feb 5 07:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.749,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.746,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.65000000000000002}]},{"osd":5,"last update":"Mon Feb 5 07:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.65900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57399999999999995,"15min":0.58199999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.74299999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.56799999999999995}]},{"osd":6,"last update":"Mon Feb 5 07:51:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.59499999999999997,"15min":0.60499999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.60499999999999998,"15min":0.61499999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.49299999999999999}]},{"osd":7,"last update":"Mon Feb 5 07:50:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.621},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55500000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717432,"statfs":{"total":95995035648,"available":95966650368,"internally_reserved":0,"allocated":716800,"data_stored":287916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.38400000000000001,"15min":0.374},"min":{"1min":0.312,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.39400000000000002},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.33100000000000002}]},{"osd":1,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.253,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.40999999999999998,"15min":0.40799999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.24299999999999999}]},{"osd":3,"last update":"Mon Feb 5 07:51:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46100000000000002,"15min":0.46899999999999997},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.316},{"interface":"front","average":{"1min":0.432,"5min":0.46400000000000002,"15min":0.47199999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.34799999999999998}]},{"osd":4,"last update":"Mon Feb 5 07:50:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.46000000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Mon Feb 5 07:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54800000000000004,"15min":0.55100000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.498}]},{"osd":6,"last update":"Mon Feb 5 07:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.55500000000000005}]},{"osd":7,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.65500000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149738,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31916,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713236,"statfs":{"total":95995035648,"available":95962353664,"internally_reserved":0,"allocated":753664,"data_stored":289059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.41299999999999998,"15min":0.42899999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.442,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40899999999999997,"15min":0.41899999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.47699999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Feb 5 07:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40999999999999998,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.51200000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.45100000000000001,"15min":0.45500000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.52000000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.51100000000000001}]},{"osd":2,"last update":"Mon Feb 5 07:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.46400000000000002,"15min":0.47499999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65700000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.436,"15min":0.45000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53700000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.55500000000000005}]},{"osd":4,"last update":"Mon Feb 5 07:50:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80500000000000005,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.64800000000000002,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.81999999999999995,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73799999999999999}]},{"osd":5,"last update":"Mon Feb 5 07:50:42 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.82799999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68600000000000005}]},{"osd":6,"last update":"Mon Feb 5 07:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.83399999999999996}]},{"osd":7,"last update":"Mon Feb 5 07:50:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.747},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.82199999999999995}]}]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27744,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717408,"statfs":{"total":95995035648,"available":95966625792,"internally_reserved":0,"allocated":741376,"data_stored":287899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79700000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.71499999999999997,"5min":0.879,"15min":0.879},"last":0.505}]},{"osd":1,"last update":"Mon Feb 5 07:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82399999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.314}]},{"osd":2,"last update":"Mon Feb 5 07:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.56200000000000006,"15min":0.57099999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65800000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.39000000000000001}]},{"osd":3,"last update":"Mon Feb 5 07:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.903,"5min":2.903,"15min":2.903},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.42199999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78600000000000003,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.439}]},{"osd":5,"last update":"Mon Feb 5 07:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.439,"15min":0.42999999999999999},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.46200000000000002,"15min":0.45200000000000001},"min":{"1min":0.35399999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.374}]},{"osd":6,"last update":"Mon Feb 5 07:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.54200000000000004}]},{"osd":7,"last update":"Mon Feb 5 07:50:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46400000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724576,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31892,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713260,"statfs":{"total":95995035648,"available":95962378240,"internally_reserved":0,"allocated":729088,"data_stored":288904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.69699999999999995}]},{"osd":1,"last update":"Mon Feb 5 07:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.68100000000000005}]},{"osd":2,"last update":"Mon Feb 5 07:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.373,"5min":0.372,"15min":0.372},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.745}]},{"osd":3,"last update":"Mon Feb 5 07:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.40600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.42099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.67200000000000004,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.77100000000000002}]},{"osd":4,"last update":"Mon Feb 5 07:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.40899999999999997,"15min":0.40200000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39000000000000001,"15min":0.38700000000000001},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63100000000000001,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.54800000000000004}]},{"osd":6,"last update":"Mon Feb 5 07:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.57299999999999995}]},{"osd":7,"last update":"Mon Feb 5 07:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.79200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-05T07:51:11.228 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-05T07:51:11.228 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-05T07:51:11.228 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-05T07:51:11.229 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 -- ceph health --format=json 2024-02-05T07:51:12.027 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:51:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:12 smithi157 ceph-mon[33345]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:12 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:13.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:51:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:51:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:12 smithi131 ceph-mon[41654]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:12 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:12 smithi131 ceph-mon[29507]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:12 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:13.655 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:13 smithi131 podman[79685]: 2024-02-05 07:51:13.545369351 +0000 UTC m=+0.128665865 volume create 0ac4d9b2e8af786708da7c15327dd294b475d1ea0074219e0af800b85d807cdd 2024-02-05T07:51:13.655 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:13 smithi131 podman[79685]: 2024-02-05 07:51:13.653938379 +0000 UTC m=+0.237234851 container create 19d0053734f7c786a34a489dfe73809d323fa5533b37a2022dcea56e1846039e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:14.550 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:14 smithi131 podman[79685]: 2024-02-05 07:51:14.268332882 +0000 UTC m=+0.851629357 container init 19d0053734f7c786a34a489dfe73809d323fa5533b37a2022dcea56e1846039e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:14.550 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:14 smithi131 podman[79685]: 2024-02-05 07:51:14.274290358 +0000 UTC m=+0.857586830 container start 19d0053734f7c786a34a489dfe73809d323fa5533b37a2022dcea56e1846039e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:14.550 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[79716]: ts=2024-02-05T07:51:14.286Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-05T07:51:14.550 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[79716]: ts=2024-02-05T07:51:14.286Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-05T07:51:14.550 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[79716]: ts=2024-02-05T07:51:14.287Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.131 port=9094 2024-02-05T07:51:14.551 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[79716]: ts=2024-02-05T07:51:14.288Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-05T07:51:14.551 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[79716]: ts=2024-02-05T07:51:14.311Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-05T07:51:14.551 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[79716]: ts=2024-02-05T07:51:14.311Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-05T07:51:14.551 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[79716]: ts=2024-02-05T07:51:14.314Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-05T07:51:14.551 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[79716]: ts=2024-02-05T07:51:14.314Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-05T07:51:14.551 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:14 smithi131 bash[79685]: 19d0053734f7c786a34a489dfe73809d323fa5533b37a2022dcea56e1846039e 2024-02-05T07:51:14.551 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:14 smithi131 systemd[1]: Started Ceph alertmanager.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T07:51:14.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.570+0000 7ffb27228700 1 -- 172.21.15.131:0/1337576999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb20107e80 msgr2=0x7ffb20108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:14.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.570+0000 7ffb27228700 1 --2- 172.21.15.131:0/1337576999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb20107e80 0x7ffb20108260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffb140098f0 tx=0x7ffb14009c00 comp rx=0 tx=0).stop 2024-02-05T07:51:14.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.570+0000 7ffb27228700 1 -- 172.21.15.131:0/1337576999 shutdown_connections 2024-02-05T07:51:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.570+0000 7ffb27228700 1 --2- 172.21.15.131:0/1337576999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb20107e80 0x7ffb20108260 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.570+0000 7ffb27228700 1 --2- 172.21.15.131:0/1337576999 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ffb20102760 0x7ffb2010ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.570+0000 7ffb27228700 1 --2- 172.21.15.131:0/1337576999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb20101da0 0x7ffb20102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.570+0000 7ffb27228700 1 -- 172.21.15.131:0/1337576999 >> 172.21.15.131:0/1337576999 conn(0x7ffb20078de0 msgr2=0x7ffb200791f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:51:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.571+0000 7ffb27228700 1 -- 172.21.15.131:0/1337576999 shutdown_connections 2024-02-05T07:51:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.571+0000 7ffb27228700 1 -- 172.21.15.131:0/1337576999 wait complete. 2024-02-05T07:51:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.571+0000 7ffb27228700 1 Processor -- start 2024-02-05T07:51:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.571+0000 7ffb27228700 1 -- start start 2024-02-05T07:51:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.571+0000 7ffb27228700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb20101da0 0x7ffb2019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:51:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.571+0000 7ffb27228700 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ffb20102760 0x7ffb2019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:51:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.571+0000 7ffb27228700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb20107e80 0x7ffb20197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:51:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.571+0000 7ffb27228700 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb20197f40 con 0x7ffb20101da0 2024-02-05T07:51:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.571+0000 7ffb27228700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb201980b0 con 0x7ffb20107e80 2024-02-05T07:51:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.571+0000 7ffb27228700 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb20198270 con 0x7ffb20102760 2024-02-05T07:51:14.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb24fc4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb20101da0 0x7ffb2019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:51:14.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb24fc4700 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb20101da0 0x7ffb2019d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39692/0 (socket says 172.21.15.131:39692) 2024-02-05T07:51:14.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb24fc4700 1 -- 172.21.15.131:0/1576268930 learned_addr learned my addr 172.21.15.131:0/1576268930 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T07:51:14.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb1ffff700 1 --2- 172.21.15.131:0/1576268930 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ffb20102760 0x7ffb2019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:51:14.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb257c5700 1 --2- 172.21.15.131:0/1576268930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb20107e80 0x7ffb20197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:51:14.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb24fc4700 1 -- 172.21.15.131:0/1576268930 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ffb20102760 msgr2=0x7ffb2019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:14.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb24fc4700 1 --2- 172.21.15.131:0/1576268930 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ffb20102760 0x7ffb2019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:14.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb24fc4700 1 -- 172.21.15.131:0/1576268930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb20107e80 msgr2=0x7ffb20197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:14.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb24fc4700 1 --2- 172.21.15.131:0/1576268930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb20107e80 0x7ffb20197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:14.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb24fc4700 1 -- 172.21.15.131:0/1576268930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb14009520 con 0x7ffb20101da0 2024-02-05T07:51:14.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb1ffff700 1 --2- 172.21.15.131:0/1576268930 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ffb20102760 0x7ffb2019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:51:14.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb24fc4700 1 --2- 172.21.15.131:0/1576268930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb20101da0 0x7ffb2019d670 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffb140059d0 tx=0x7ffb1400f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:51:14.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb1dffb700 1 -- 172.21.15.131:0/1576268930 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1401c070 con 0x7ffb20101da0 2024-02-05T07:51:14.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb27228700 1 -- 172.21.15.131:0/1576268930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb201984f0 con 0x7ffb20101da0 2024-02-05T07:51:14.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb1dffb700 1 -- 172.21.15.131:0/1576268930 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7ffb14005040 con 0x7ffb20101da0 2024-02-05T07:51:14.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb1dffb700 1 -- 172.21.15.131:0/1576268930 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14004b40 con 0x7ffb20101da0 2024-02-05T07:51:14.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.572+0000 7ffb27228700 1 -- 172.21.15.131:0/1576268930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb201a45f0 con 0x7ffb20101da0 2024-02-05T07:51:14.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.573+0000 7ffb27228700 1 -- 172.21.15.131:0/1576268930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb20103460 con 0x7ffb20101da0 2024-02-05T07:51:14.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.574+0000 7ffb1dffb700 1 -- 172.21.15.131:0/1576268930 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7ffb14004ca0 con 0x7ffb20101da0 2024-02-05T07:51:14.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.574+0000 7ffb1dffb700 1 --2- 172.21.15.131:0/1576268930 >> v2:172.21.15.131:6800/2614238408 conn(0x7ffb1006da80 0x7ffb1006ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:51:14.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.574+0000 7ffb1dffb700 1 -- 172.21.15.131:0/1576268930 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (secure 0 0 0) 0x7ffb1408e5d0 con 0x7ffb20101da0 2024-02-05T07:51:14.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.574+0000 7ffb1ffff700 1 --2- 172.21.15.131:0/1576268930 >> v2:172.21.15.131:6800/2614238408 conn(0x7ffb1006da80 0x7ffb1006ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:51:14.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.575+0000 7ffb1ffff700 1 --2- 172.21.15.131:0/1576268930 >> v2:172.21.15.131:6800/2614238408 conn(0x7ffb1006da80 0x7ffb1006ff40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffb20198f00 tx=0x7ffb0c0073d0 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:51:14.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.579+0000 7ffb1dffb700 1 -- 172.21.15.131:0/1576268930 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffb1405baa0 con 0x7ffb20101da0 2024-02-05T07:51:14.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.820+0000 7ffb27228700 1 -- 172.21.15.131:0/1576268930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ffb200689e0 con 0x7ffb20101da0 2024-02-05T07:51:14.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.820+0000 7ffb1dffb700 1 -- 172.21.15.131:0/1576268930 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ffb1405b630 con 0x7ffb20101da0 2024-02-05T07:51:14.821 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:14] ENGINE Bus STOPPING 2024-02-05T07:51:14.821 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:14] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-05T07:51:14.822 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:14] ENGINE Bus STOPPED 2024-02-05T07:51:14.822 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:14] ENGINE Bus STARTING 2024-02-05T07:51:14.822 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:14] ENGINE Serving on http://:::9283 2024-02-05T07:51:14.822 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:14] ENGINE Bus STARTED 2024-02-05T07:51:14.822 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-02-05T07:51:14.822 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-05T07:51:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.823+0000 7ffb27228700 1 -- 172.21.15.131:0/1576268930 >> v2:172.21.15.131:6800/2614238408 conn(0x7ffb1006da80 msgr2=0x7ffb1006ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.823+0000 7ffb27228700 1 --2- 172.21.15.131:0/1576268930 >> v2:172.21.15.131:6800/2614238408 conn(0x7ffb1006da80 0x7ffb1006ff40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffb20198f00 tx=0x7ffb0c0073d0 comp rx=0 tx=0).stop 2024-02-05T07:51:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.823+0000 7ffb27228700 1 -- 172.21.15.131:0/1576268930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb20101da0 msgr2=0x7ffb2019d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:51:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.823+0000 7ffb27228700 1 --2- 172.21.15.131:0/1576268930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb20101da0 0x7ffb2019d670 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffb140059d0 tx=0x7ffb1400f9e0 comp rx=0 tx=0).stop 2024-02-05T07:51:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.823+0000 7ffb27228700 1 -- 172.21.15.131:0/1576268930 shutdown_connections 2024-02-05T07:51:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.823+0000 7ffb27228700 1 --2- 172.21.15.131:0/1576268930 >> v2:172.21.15.131:6800/2614238408 conn(0x7ffb1006da80 0x7ffb1006ff40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.823+0000 7ffb27228700 1 --2- 172.21.15.131:0/1576268930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb20101da0 0x7ffb2019d670 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.823+0000 7ffb27228700 1 --2- 172.21.15.131:0/1576268930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb20107e80 0x7ffb20197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.823+0000 7ffb27228700 1 --2- 172.21.15.131:0/1576268930 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ffb20102760 0x7ffb2019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:51:14.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.823+0000 7ffb27228700 1 -- 172.21.15.131:0/1576268930 >> 172.21.15.131:0/1576268930 conn(0x7ffb20078de0 msgr2=0x7ffb20105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:51:14.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.823+0000 7ffb27228700 1 -- 172.21.15.131:0/1576268930 shutdown_connections 2024-02-05T07:51:14.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-02-05T07:51:14.823+0000 7ffb27228700 1 -- 172.21.15.131:0/1576268930 wait complete. 2024-02-05T07:51:15.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[29507]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:15.140 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T07:51:15.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.141 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1576268930' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-05T07:51:15.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[41654]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:15.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.141 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T07:51:15.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.142 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1576268930' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-05T07:51:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:14 smithi157 ceph-mon[33345]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:14 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:14 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:14 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:14 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:14 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:14 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:14 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T07:51:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:14 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1576268930' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-05T07:51:15.529 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-05T07:51:15.529 INFO:tasks.cephadm:Setup complete, yielding 2024-02-05T07:51:15.529 INFO:teuthology.run_tasks:Running task workunit... 2024-02-05T07:51:15.544 INFO:tasks.workunit:Pulling workunits from ref 164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:51:15.545 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-05T07:51:15.545 DEBUG:teuthology.orchestra.run.smithi131:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-05T07:51:15.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T07:51:15.561 INFO:teuthology.orchestra.run.smithi131.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-02-05T07:51:15.561 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-02-05T07:51:15.617 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-02-05T07:51:15.617 DEBUG:teuthology.orchestra.run.smithi131:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-02-05T07:51:15.672 INFO:tasks.workunit:timeout=3h 2024-02-05T07:51:15.672 INFO:tasks.workunit:cleanup=True 2024-02-05T07:51:15.672 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:51:15.727 INFO:tasks.workunit.client.0.smithi131.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-05T07:51:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:15 smithi157 ceph-mon[33345]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-05T07:51:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:15 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T07:51:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:15 smithi157 ceph-mon[33345]: Deploying daemon grafana.a on smithi157 2024-02-05T07:51:16.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:15 smithi131 ceph-mon[29507]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-05T07:51:16.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:15 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T07:51:16.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:15 smithi131 ceph-mon[29507]: Deploying daemon grafana.a on smithi157 2024-02-05T07:51:16.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:15 smithi131 ceph-mon[41654]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-05T07:51:16.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:15 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T07:51:16.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:15 smithi131 ceph-mon[41654]: Deploying daemon grafana.a on smithi157 2024-02-05T07:51:16.768 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:16 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[79716]: ts=2024-02-05T07:51:16.288Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000146132s 2024-02-05T07:51:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:16 smithi157 ceph-mon[33345]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:16 smithi131 ceph-mon[41654]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:17.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:16 smithi131 ceph-mon[29507]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:18.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:18 smithi157 ceph-mon[33345]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:18.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:18 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:18.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:51:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:51:18.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:18 smithi131 ceph-mon[41654]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:18.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:18 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:18.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:18 smithi131 ceph-mon[29507]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:18.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:18 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:19.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:19 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:19.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:19 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:19.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:19 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:20.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:20 smithi157 ceph-mon[33345]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:20.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:20 smithi131 ceph-mon[29507]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:20.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:20 smithi131 ceph-mon[41654]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:22.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:22 smithi157 ceph-mon[33345]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:22.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:22 smithi131 ceph-mon[41654]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:22.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:22 smithi131 ceph-mon[29507]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:23.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:51:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:51:24.768 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:24 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[79716]: ts=2024-02-05T07:51:24.290Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.00223268s 2024-02-05T07:51:25.199 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:24 smithi157 ceph-mon[33345]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:25.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:24 smithi131 ceph-mon[41654]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:25.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:24 smithi131 ceph-mon[29507]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:26 smithi157 ceph-mon[33345]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:26 smithi131 ceph-mon[41654]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:27.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:26 smithi131 ceph-mon[29507]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:27.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:28.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:28.319 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 podman[67150]: 2024-02-05 07:51:28.106747973 +0000 UTC m=+0.172224357 container create c01c9552d28da0c1a17a32a9d0ac238607a6de8c041d4dabc1c2d8ee5a5606c9 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a, release=239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, summary=Grafana Container configured for Ceph mgr/dashboard integration, version=8.5, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, name=ubi8, io.openshift.tags=base rhel8, description=Ceph Grafana Container, distribution-scope=public, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, io.buildah.version=1.32.0, vendor=Red Hat, Inc., maintainer=Paul Cuzner , com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=, architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.component=ubi8-container) 2024-02-05T07:51:28.569 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:51:28.571 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 podman[67150]: 2024-02-05 07:51:28.34220811 +0000 UTC m=+0.407684448 container init c01c9552d28da0c1a17a32a9d0ac238607a6de8c041d4dabc1c2d8ee5a5606c9 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a, distribution-scope=public, version=8.5, maintainer=Paul Cuzner , summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0, description=Ceph Grafana Container, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, architecture=x86_64, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.component=ubi8-container, vendor=Red Hat, Inc., io.openshift.tags=base rhel8, build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, vcs-type=git, name=ubi8) 2024-02-05T07:51:28.571 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 podman[67150]: 2024-02-05 07:51:28.347842222 +0000 UTC m=+0.413318560 container start c01c9552d28da0c1a17a32a9d0ac238607a6de8c041d4dabc1c2d8ee5a5606c9 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a, name=ubi8, maintainer=Paul Cuzner , com.redhat.component=ubi8-container, summary=Grafana Container configured for Ceph mgr/dashboard integration, description=Ceph Grafana Container, release=239.1651231664, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, version=8.5, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, build-date=2022-04-29T11:48:11.055897, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.expose-services=, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vendor=Red Hat, Inc., distribution-scope=public, io.buildah.version=1.32.0, architecture=x86_64, vcs-type=git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-02-05T07:51:28.572 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 bash[67150]: c01c9552d28da0c1a17a32a9d0ac238607a6de8c041d4dabc1c2d8ee5a5606c9 2024-02-05T07:51:28.572 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 systemd[1]: Started Ceph grafana.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T07:51:28.572 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=settings t=2024-02-05T07:51:28.509273575Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-02-05T07:51:28.572 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=settings t=2024-02-05T07:51:28.509476941Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-02-05T07:51:28.572 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=settings t=2024-02-05T07:51:28.509494699Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-02-05T07:51:28.572 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=settings t=2024-02-05T07:51:28.509501274Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-02-05T07:51:28.573 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=settings t=2024-02-05T07:51:28.509507348Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-05T07:51:28.573 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=settings t=2024-02-05T07:51:28.509520572Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-05T07:51:28.573 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=settings t=2024-02-05T07:51:28.50952612Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-05T07:51:28.573 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=settings t=2024-02-05T07:51:28.509531808Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-05T07:51:28.573 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=settings t=2024-02-05T07:51:28.509537657Z level=info msg="Path Home" path=/usr/share/grafana 2024-02-05T07:51:28.573 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=settings t=2024-02-05T07:51:28.50954371Z level=info msg="Path Data" path=/var/lib/grafana 2024-02-05T07:51:28.574 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=settings t=2024-02-05T07:51:28.509549169Z level=info msg="Path Logs" path=/var/log/grafana 2024-02-05T07:51:28.574 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=settings t=2024-02-05T07:51:28.509554799Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-02-05T07:51:28.574 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=settings t=2024-02-05T07:51:28.509560262Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-02-05T07:51:28.574 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=settings t=2024-02-05T07:51:28.509565923Z level=info msg="App mode production" 2024-02-05T07:51:28.574 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=sqlstore t=2024-02-05T07:51:28.509608127Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-02-05T07:51:28.574 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=sqlstore t=2024-02-05T07:51:28.509627166Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-05T07:51:28.575 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.510517965Z level=info msg="Starting DB migrations" 2024-02-05T07:51:28.575 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.510595686Z level=info msg="Executing migration" id="create migration_log table" 2024-02-05T07:51:28.575 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.511101326Z level=info msg="Executing migration" id="create user table" 2024-02-05T07:51:28.575 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.511517538Z level=info msg="Executing migration" id="add unique index user.login" 2024-02-05T07:51:28.575 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.512049972Z level=info msg="Executing migration" id="add unique index user.email" 2024-02-05T07:51:28.575 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.512450227Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-02-05T07:51:28.575 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.512877884Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-02-05T07:51:28.575 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.513259957Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-02-05T07:51:28.575 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.514018754Z level=info msg="Executing migration" id="create user table v2" 2024-02-05T07:51:28.576 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.514407063Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-02-05T07:51:28.576 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.514801475Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-02-05T07:51:28.576 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.515219367Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-02-05T07:51:28.576 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.515521151Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-02-05T07:51:28.576 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.515875017Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-02-05T07:51:28.576 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.516248555Z level=info msg="Executing migration" id="Update user table charset" 2024-02-05T07:51:28.577 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.516503677Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-02-05T07:51:28.577 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.516943632Z level=info msg="Executing migration" id="Add missing user data" 2024-02-05T07:51:28.577 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.517247827Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-02-05T07:51:28.577 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.51766798Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-02-05T07:51:28.577 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.51805326Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-02-05T07:51:28.577 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.518427818Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-02-05T07:51:28.578 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.519523847Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-02-05T07:51:28.578 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.519910671Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-02-05T07:51:28.578 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.520292683Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-02-05T07:51:28.578 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.520676147Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-02-05T07:51:28.578 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.521054906Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-02-05T07:51:28.578 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.52141773Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-02-05T07:51:28.579 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.521679189Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-02-05T07:51:28.579 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.522021668Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-02-05T07:51:28.579 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.522336121Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-02-05T07:51:28.579 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.522699167Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-02-05T07:51:28.579 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.523045782Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-05T07:51:28.579 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.523646212Z level=info msg="Executing migration" id="create temp_user v2" 2024-02-05T07:51:28.579 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.524002317Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-02-05T07:51:28.580 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.524323577Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-02-05T07:51:28.580 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.524678705Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-02-05T07:51:28.580 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.525037898Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-02-05T07:51:28.580 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.525411134Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-02-05T07:51:28.580 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.525714908Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-02-05T07:51:28.580 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.526013479Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-02-05T07:51:28.581 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.526278307Z level=info msg="Executing migration" id="create star table" 2024-02-05T07:51:28.581 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.526612812Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-02-05T07:51:28.581 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.527058562Z level=info msg="Executing migration" id="create org table v1" 2024-02-05T07:51:28.581 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.527453517Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-02-05T07:51:28.581 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.527818891Z level=info msg="Executing migration" id="create org_user table v1" 2024-02-05T07:51:28.581 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.528168885Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-02-05T07:51:28.582 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.52853171Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-02-05T07:51:28.582 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.528921048Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-02-05T07:51:28.582 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.52926787Z level=info msg="Executing migration" id="Update org table charset" 2024-02-05T07:51:28.582 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.529490966Z level=info msg="Executing migration" id="Update org_user table charset" 2024-02-05T07:51:28.582 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.529776251Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-02-05T07:51:28.582 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.530022814Z level=info msg="Executing migration" id="create dashboard table" 2024-02-05T07:51:28.583 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.530350373Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-02-05T07:51:28.583 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.530775373Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-02-05T07:51:28.583 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.531166858Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-02-05T07:51:28.583 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.531504965Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-02-05T07:51:28.583 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.531906045Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-05T07:51:28.583 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.532221752Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-02-05T07:51:28.583 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.533063569Z level=info msg="Executing migration" id="create dashboard v2" 2024-02-05T07:51:28.584 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.533417722Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-02-05T07:51:28.584 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.533794389Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-02-05T07:51:28.584 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.534189767Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-02-05T07:51:28.584 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.534459507Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-02-05T07:51:28.584 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.534876626Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-02-05T07:51:28.584 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.535114052Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-02-05T07:51:28.585 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.535536878Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-02-05T07:51:28.585 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.536030088Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-02-05T07:51:28.585 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.536443025Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-02-05T07:51:28.585 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.536825063Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-02-05T07:51:28.585 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.53725845Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-02-05T07:51:28.585 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.537643345Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-02-05T07:51:28.586 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.538056529Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-02-05T07:51:28.586 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.53828115Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-02-05T07:51:28.586 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.538497922Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-02-05T07:51:28.586 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.53906717Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-02-05T07:51:28.586 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.539549464Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-02-05T07:51:28.586 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.540074275Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-02-05T07:51:28.586 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.540527139Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-02-05T07:51:28.587 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.540828888Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-02-05T07:51:28.587 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.541227405Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-02-05T07:51:28.587 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.541630615Z level=info msg="Executing migration" id="Update dashboard title length" 2024-02-05T07:51:28.587 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.541908217Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-05T07:51:28.587 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.542284517Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-02-05T07:51:28.587 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.542650308Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-05T07:51:28.587 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.543566777Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-02-05T07:51:28.589 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.543959671Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-05T07:51:28.590 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.544330282Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-05T07:51:28.590 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.544714288Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-02-05T07:51:28.590 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.545002703Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-02-05T07:51:28.590 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.545286825Z level=info msg="Executing migration" id="Add check_sum column" 2024-02-05T07:51:28.590 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.545772752Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-02-05T07:51:28.590 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.546140745Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-02-05T07:51:28.591 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.546377141Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-02-05T07:51:28.591 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.546651433Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-02-05T07:51:28.591 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.547044562Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-02-05T07:51:28.591 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.547504712Z level=info msg="Executing migration" id="create data_source table" 2024-02-05T07:51:28.591 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.547894524Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-02-05T07:51:28.591 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.548278159Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-02-05T07:51:28.591 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.548650103Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-02-05T07:51:28.592 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.549026267Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-02-05T07:51:28.592 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.54935406Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-02-05T07:51:28.592 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.550343834Z level=info msg="Executing migration" id="create data_source table v2" 2024-02-05T07:51:28.592 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.550733768Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-02-05T07:51:28.592 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.551113354Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-02-05T07:51:28.592 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.551474811Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-02-05T07:51:28.593 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.551807805Z level=info msg="Executing migration" id="Add column with_credentials" 2024-02-05T07:51:28.593 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.552266025Z level=info msg="Executing migration" id="Add secure json data column" 2024-02-05T07:51:28.593 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.552753634Z level=info msg="Executing migration" id="Update data_source table charset" 2024-02-05T07:51:28.593 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.553011386Z level=info msg="Executing migration" id="Update initial version to 1" 2024-02-05T07:51:28.593 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.553255869Z level=info msg="Executing migration" id="Add read_only data column" 2024-02-05T07:51:28.593 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.553794127Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-02-05T07:51:28.594 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.554075955Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-02-05T07:51:28.594 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.554318701Z level=info msg="Executing migration" id="Add uid column" 2024-02-05T07:51:28.594 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.554804422Z level=info msg="Executing migration" id="Update uid value" 2024-02-05T07:51:28.594 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.555078407Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-02-05T07:51:28.594 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.555446771Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-02-05T07:51:28.594 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.555832779Z level=info msg="Executing migration" id="create api_key table" 2024-02-05T07:51:28.594 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.556231671Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-02-05T07:51:28.594 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.556644213Z level=info msg="Executing migration" id="add index api_key.key" 2024-02-05T07:51:28.594 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.557019784Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-02-05T07:51:28.594 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.557416179Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-02-05T07:51:28.594 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.557775133Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-02-05T07:51:28.594 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.558132458Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-02-05T07:51:28.594 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.558471785Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-02-05T07:51:28.595 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.559457078Z level=info msg="Executing migration" id="create api_key table v2" 2024-02-05T07:51:28.595 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.559827544Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-02-05T07:51:28.595 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.560168144Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-02-05T07:51:28.595 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.56055338Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-02-05T07:51:28.595 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.560930537Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-02-05T07:51:28.595 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.561196709Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-02-05T07:51:28.595 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.561478706Z level=info msg="Executing migration" id="Update api_key table charset" 2024-02-05T07:51:28.595 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.561755492Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-02-05T07:51:28.595 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.562229431Z level=info msg="Executing migration" id="Add service account foreign key" 2024-02-05T07:51:28.595 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.562744784Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-02-05T07:51:28.595 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.563022098Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-02-05T07:51:28.595 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.56349146Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-02-05T07:51:28.595 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.564006215Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-02-05T07:51:28.595 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.564328615Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-02-05T07:51:28.595 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.564662181Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-02-05T07:51:28.595 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.565022581Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-02-05T07:51:28.596 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.565418978Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-05T07:51:28.596 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.565801641Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-05T07:51:28.596 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.566207111Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-02-05T07:51:28.596 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.566435152Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-02-05T07:51:28.596 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.566701252Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-02-05T07:51:28.596 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.567263698Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-02-05T07:51:28.596 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.567782635Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-05T07:51:28.596 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.568054088Z level=info msg="Executing migration" id="create quota table v1" 2024-02-05T07:51:28.596 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.568402596Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-05T07:51:28.596 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.568798772Z level=info msg="Executing migration" id="Update quota table charset" 2024-02-05T07:51:28.596 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.569083051Z level=info msg="Executing migration" id="create plugin_setting table" 2024-02-05T07:51:28.596 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.569504446Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-05T07:51:28.596 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.569970865Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-02-05T07:51:28.596 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.570653987Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-02-05T07:51:28.597 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.570979778Z level=info msg="Executing migration" id="create session table" 2024-02-05T07:51:28.597 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.571426172Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-02-05T07:51:28.597 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.571750127Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-02-05T07:51:28.597 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.572042255Z level=info msg="Executing migration" id="create playlist table v2" 2024-02-05T07:51:28.597 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.57244979Z level=info msg="Executing migration" id="create playlist item table v2" 2024-02-05T07:51:28.597 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.572872183Z level=info msg="Executing migration" id="Update playlist table charset" 2024-02-05T07:51:28.597 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.573123257Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-02-05T07:51:28.840 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.573388249Z level=info msg="Executing migration" id="drop preferences table v2" 2024-02-05T07:51:28.840 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.573746818Z level=info msg="Executing migration" id="drop preferences table v3" 2024-02-05T07:51:28.840 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.574076452Z level=info msg="Executing migration" id="create preferences table v3" 2024-02-05T07:51:28.840 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.57448395Z level=info msg="Executing migration" id="Update preferences table charset" 2024-02-05T07:51:28.840 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.574791001Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-02-05T07:51:28.840 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.575352631Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-02-05T07:51:28.841 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.575653136Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-02-05T07:51:28.841 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.576207503Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-02-05T07:51:28.841 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.576817283Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-02-05T07:51:28.841 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.577094197Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-02-05T07:51:28.841 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.577501453Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-02-05T07:51:28.841 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.57795319Z level=info msg="Executing migration" id="create alert table v1" 2024-02-05T07:51:28.841 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.578404281Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-02-05T07:51:28.842 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.578839889Z level=info msg="Executing migration" id="add index alert state" 2024-02-05T07:51:28.842 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.579233489Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-02-05T07:51:28.842 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.579623489Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-02-05T07:51:28.842 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.5799779Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-05T07:51:28.842 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.580398676Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-05T07:51:28.842 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.580773911Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-05T07:51:28.843 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.58214069Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-02-05T07:51:28.843 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.582493723Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-05T07:51:28.843 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.582914702Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-02-05T07:51:28.843 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.583180573Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-02-05T07:51:28.843 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.583478299Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-02-05T07:51:28.843 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.583840477Z level=info msg="Executing migration" id="Add column is_default" 2024-02-05T07:51:28.843 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.584416521Z level=info msg="Executing migration" id="Add column frequency" 2024-02-05T07:51:28.844 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.585023592Z level=info msg="Executing migration" id="Add column send_reminder" 2024-02-05T07:51:28.844 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.585661583Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-02-05T07:51:28.844 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.58624371Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-02-05T07:51:28.844 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.586642327Z level=info msg="Executing migration" id="Update alert table charset" 2024-02-05T07:51:28.844 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.58690359Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-02-05T07:51:28.844 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.587129057Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-02-05T07:51:28.844 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.587479104Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-05T07:51:28.844 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.587885847Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-02-05T07:51:28.844 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.588213667Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-02-05T07:51:28.844 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.588542203Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-05T07:51:28.844 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.588958899Z level=info msg="Executing migration" id="Add for to alert table" 2024-02-05T07:51:28.844 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.589524027Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-02-05T07:51:28.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.590133508Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-02-05T07:51:28.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.590400885Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-02-05T07:51:28.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.590852347Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-02-05T07:51:28.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.591404809Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-02-05T07:51:28.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.592038225Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-02-05T07:51:28.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.592302062Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-02-05T07:51:28.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.592694906Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-02-05T07:51:28.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.593068707Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-02-05T07:51:28.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.593312849Z level=info msg="Executing migration" id="create annotation table v5" 2024-02-05T07:51:28.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.593721508Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-02-05T07:51:28.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.594117483Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-02-05T07:51:28.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.594488155Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-02-05T07:51:28.845 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.594890887Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-02-05T07:51:28.846 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.595320703Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-02-05T07:51:28.846 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.59576709Z level=info msg="Executing migration" id="Update annotation table charset" 2024-02-05T07:51:28.846 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.596019271Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-02-05T07:51:28.846 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.596637662Z level=info msg="Executing migration" id="Drop category_id index" 2024-02-05T07:51:28.846 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.597007754Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-02-05T07:51:28.846 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.597619655Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-02-05T07:51:28.848 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.597994111Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-05T07:51:28.848 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.598368764Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-05T07:51:28.848 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.598740751Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-05T07:51:28.848 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.600247162Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-02-05T07:51:28.848 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.600595086Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-05T07:51:28.848 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.600980311Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-02-05T07:51:28.849 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.601228479Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-02-05T07:51:28.849 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.60152175Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-02-05T07:51:28.849 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.601807577Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-02-05T07:51:28.849 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.602411865Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-02-05T07:51:28.849 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.603187369Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-02-05T07:51:28.849 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.603688828Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-02-05T07:51:28.849 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.604142249Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-02-05T07:51:28.849 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.604424712Z level=info msg="Executing migration" id="Add epoch_end column" 2024-02-05T07:51:28.849 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.605188217Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-02-05T07:51:28.849 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.605641007Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-02-05T07:51:28.849 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.60597107Z level=info msg="Executing migration" id="Move region to single row" 2024-02-05T07:51:28.849 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.606328611Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-02-05T07:51:28.849 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.606759527Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-05T07:51:28.850 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.607214681Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-05T07:51:28.850 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.607569804Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-05T07:51:28.850 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.60795516Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-05T07:51:28.850 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.60830303Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-02-05T07:51:28.850 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.608688658Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-02-05T07:51:28.850 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.608959115Z level=info msg="Executing migration" id="create test_data table" 2024-02-05T07:51:28.850 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.609340875Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-02-05T07:51:28.850 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.609744932Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-02-05T07:51:28.850 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.610130106Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-05T07:51:28.851 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.610503098Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-02-05T07:51:28.851 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.61078485Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-02-05T07:51:28.851 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.611037706Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-02-05T07:51:28.851 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.611258332Z level=info msg="Executing migration" id="create team table" 2024-02-05T07:51:28.851 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.611616439Z level=info msg="Executing migration" id="add index team.org_id" 2024-02-05T07:51:28.851 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.61203111Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-02-05T07:51:28.851 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.612412927Z level=info msg="Executing migration" id="create team member table" 2024-02-05T07:51:28.851 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.612771516Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-02-05T07:51:28.851 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.613151707Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-02-05T07:51:28.851 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.613542746Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-02-05T07:51:28.851 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.613919485Z level=info msg="Executing migration" id="Add column email to team table" 2024-02-05T07:51:28.851 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.614542699Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-02-05T07:51:28.851 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.615199293Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-02-05T07:51:28.851 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.61586389Z level=info msg="Executing migration" id="create dashboard acl table" 2024-02-05T07:51:28.852 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.616223123Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-02-05T07:51:28.852 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.616634684Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-05T07:51:28.852 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.617052987Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-05T07:51:28.852 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.617432836Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-02-05T07:51:28.852 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.617885997Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-02-05T07:51:28.852 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.61826736Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-02-05T07:51:28.852 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.618661176Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-02-05T07:51:28.852 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.61903641Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-02-05T07:51:28.852 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.619366934Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-02-05T07:51:28.852 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.619650234Z level=info msg="Executing migration" id="create tag table" 2024-02-05T07:51:28.852 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.620024551Z level=info msg="Executing migration" id="add index tag.key_value" 2024-02-05T07:51:28.852 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.620403098Z level=info msg="Executing migration" id="create login attempt table" 2024-02-05T07:51:28.852 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.62075608Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-02-05T07:51:28.853 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.621132193Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-02-05T07:51:28.853 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.621467888Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-05T07:51:28.853 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.623345894Z level=info msg="Executing migration" id="create login_attempt v2" 2024-02-05T07:51:28.853 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.623707081Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-02-05T07:51:28.853 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.624094197Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-02-05T07:51:28.853 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.62434668Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-02-05T07:51:28.853 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.624678574Z level=info msg="Executing migration" id="create user auth table" 2024-02-05T07:51:28.853 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.625019663Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-05T07:51:28.853 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.625404133Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-02-05T07:51:28.853 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.625669186Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-02-05T07:51:28.853 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.626374528Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-02-05T07:51:28.853 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.627097686Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-02-05T07:51:28.853 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.627804677Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-02-05T07:51:28.853 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.628505983Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-02-05T07:51:28.853 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.628913909Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-02-05T07:51:28.854 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.629585639Z level=info msg="Executing migration" id="create server_lock table" 2024-02-05T07:51:28.854 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.629959483Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-02-05T07:51:28.854 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.630343252Z level=info msg="Executing migration" id="create user auth token table" 2024-02-05T07:51:28.854 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.630731587Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-02-05T07:51:28.854 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.631113263Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-02-05T07:51:28.854 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.631486912Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-02-05T07:51:28.854 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.631910895Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-02-05T07:51:28.854 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.632656563Z level=info msg="Executing migration" id="create cache_data table" 2024-02-05T07:51:28.854 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.633032841Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-02-05T07:51:28.855 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.633418982Z level=info msg="Executing migration" id="create short_url table v1" 2024-02-05T07:51:28.855 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.63380424Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-02-05T07:51:28.855 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.634185439Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-02-05T07:51:28.855 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.634410008Z level=info msg="Executing migration" id="delete alert_definition table" 2024-02-05T07:51:28.855 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.634676544Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-02-05T07:51:28.855 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.635071019Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-02-05T07:51:28.855 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.635445691Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-02-05T07:51:28.855 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.635853834Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-02-05T07:51:28.855 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.636108111Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-02-05T07:51:28.855 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.636449926Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-02-05T07:51:28.855 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.636824037Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-02-05T07:51:28.855 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.637208574Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-02-05T07:51:28.855 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.637597026Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-02-05T07:51:28.856 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.638334537Z level=info msg="Executing migration" id="drop alert_definition table" 2024-02-05T07:51:28.856 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.638728181Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-02-05T07:51:28.856 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.638994054Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-02-05T07:51:28.856 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.63932383Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-05T07:51:28.856 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.639732577Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-05T07:51:28.856 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.640116537Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-05T07:51:28.856 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.640341599Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-02-05T07:51:28.856 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.640734951Z level=info msg="Executing migration" id="create alert_instance table" 2024-02-05T07:51:28.856 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.641115266Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-05T07:51:28.856 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.64146847Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-05T07:51:28.857 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.641857374Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-02-05T07:51:28.857 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.642566987Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-05T07:51:28.857 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.642939736Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-02-05T07:51:28.857 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.643284923Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-02-05T07:51:28.857 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.646599601Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-02-05T07:51:28.857 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.649904053Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-05T07:51:28.857 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.650357706Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-02-05T07:51:28.857 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.650777748Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-02-05T07:51:28.857 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.651576997Z level=info msg="Executing migration" id="create alert_rule table" 2024-02-05T07:51:28.858 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.651964858Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-02-05T07:51:28.858 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.652354415Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-02-05T07:51:28.858 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.65275134Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-05T07:51:28.858 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.653137776Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-02-05T07:51:28.858 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.653367657Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-02-05T07:51:28.858 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.654162133Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-02-05T07:51:28.858 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.654968022Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-02-05T07:51:28.858 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.6557658Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-02-05T07:51:28.858 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.656150013Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-05T07:51:28.858 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.656523887Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-02-05T07:51:28.858 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.657289317Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-02-05T07:51:28.858 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.658094058Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-05T07:51:28.858 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.658496407Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-02-05T07:51:28.859 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.65962506Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-02-05T07:51:28.859 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.660870143Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-02-05T07:51:28.859 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.661204701Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-02-05T07:51:28.859 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.661662875Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-05T07:51:28.859 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.662077245Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-05T07:51:28.859 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.662486299Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-05T07:51:28.859 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.662760344Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-02-05T07:51:28.859 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.663558239Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-02-05T07:51:28.859 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.664382739Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-02-05T07:51:28.859 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.665205417Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-02-05T07:51:28.859 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.666120569Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-02-05T07:51:28.859 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.666937727Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-02-05T07:51:28.859 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.66723839Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-02-05T07:51:28.860 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.667614193Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-02-05T07:51:28.860 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.668442464Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-05T07:51:28.860 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.668730206Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-02-05T07:51:28.860 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.669611185Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-02-05T07:51:28.860 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.670041327Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-02-05T07:51:28.860 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.670871055Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-02-05T07:51:28.860 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.671234061Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-02-05T07:51:28.860 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.671691509Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-02-05T07:51:28.860 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.672531273Z level=info msg="Executing migration" id="create provenance_type table" 2024-02-05T07:51:28.860 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.672952468Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-02-05T07:51:28.860 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.673350852Z level=info msg="Executing migration" id="create alert_image table" 2024-02-05T07:51:28.860 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.673736374Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-02-05T07:51:28.860 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.674147881Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-02-05T07:51:28.861 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.674386116Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-02-05T07:51:28.861 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.674769089Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-02-05T07:51:28.861 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.67518621Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-02-05T07:51:28.861 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.675265529Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-02-05T07:51:28.861 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.675470248Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-02-05T07:51:28.861 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.675795583Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-02-05T07:51:28.861 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.676145097Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-02-05T07:51:28.861 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.676257467Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-02-05T07:51:28.861 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.676385671Z level=info msg="alerts found to migrate" alerts=0 2024-02-05T07:51:28.861 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.676747161Z level=info msg="Executing migration" id="create library_element table v1" 2024-02-05T07:51:28.861 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.677179831Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-02-05T07:51:28.861 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.677605102Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-02-05T07:51:28.861 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.677988897Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-02-05T07:51:28.861 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.678415413Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-02-05T07:51:28.861 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.678872394Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-02-05T07:51:28.862 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.679119429Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-02-05T07:51:28.862 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.679412225Z level=info msg="Executing migration" id="create data_keys table" 2024-02-05T07:51:28.862 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.67979881Z level=info msg="Executing migration" id="create secrets table" 2024-02-05T07:51:28.862 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.680190722Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-02-05T07:51:28.862 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.684453201Z level=info msg="Executing migration" id="add name column into data_keys" 2024-02-05T07:51:28.862 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.685331897Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-02-05T07:51:28.862 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.685581053Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-02-05T07:51:28.862 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.689867316Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-02-05T07:51:28.862 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.694141887Z level=info msg="Executing migration" id="create kv_store table v1" 2024-02-05T07:51:28.862 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.694537718Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-02-05T07:51:28.862 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.694958333Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-02-05T07:51:28.862 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.69524175Z level=info msg="Executing migration" id="create permission table" 2024-02-05T07:51:28.862 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.695627641Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-02-05T07:51:28.862 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.696057371Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-02-05T07:51:28.862 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.696486679Z level=info msg="Executing migration" id="create role table" 2024-02-05T07:51:28.862 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.69689292Z level=info msg="Executing migration" id="add column display_name" 2024-02-05T07:51:28.863 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.697801944Z level=info msg="Executing migration" id="add column group_name" 2024-02-05T07:51:28.863 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.698705349Z level=info msg="Executing migration" id="add index role.org_id" 2024-02-05T07:51:28.863 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.699326668Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-02-05T07:51:28.863 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.699753763Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-02-05T07:51:28.863 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.700181896Z level=info msg="Executing migration" id="create team role table" 2024-02-05T07:51:28.863 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.700550114Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-02-05T07:51:28.863 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.700984914Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-02-05T07:51:28.863 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.701435789Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-02-05T07:51:28.863 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.701872056Z level=info msg="Executing migration" id="create user role table" 2024-02-05T07:51:28.863 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.702260541Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-02-05T07:51:28.863 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.702712977Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-02-05T07:51:28.863 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.703148147Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-02-05T07:51:28.863 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.703573824Z level=info msg="Executing migration" id="create builtin role table" 2024-02-05T07:51:28.863 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.703994394Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-02-05T07:51:28.863 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.704392539Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-02-05T07:51:28.864 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.704795713Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-02-05T07:51:28.864 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.705778194Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-02-05T07:51:28.864 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.706246494Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-02-05T07:51:28.864 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.706661448Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-02-05T07:51:28.864 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.707077792Z level=info msg="Executing migration" id="add unique index role.uid" 2024-02-05T07:51:28.864 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.707457088Z level=info msg="Executing migration" id="create seed assignment table" 2024-02-05T07:51:28.864 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.707837521Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-02-05T07:51:28.864 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.708262297Z level=info msg="Executing migration" id="add column hidden to role table" 2024-02-05T07:51:28.864 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.709238927Z level=info msg="Executing migration" id="create query_history table v1" 2024-02-05T07:51:28.864 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.709628804Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-02-05T07:51:28.865 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.710066066Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-02-05T07:51:28.865 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.710308029Z level=info msg="Executing migration" id="teams permissions migration" 2024-02-05T07:51:28.865 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.710701724Z level=info msg="Executing migration" id="dashboard permissions" 2024-02-05T07:51:28.865 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.711160317Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-02-05T07:51:28.865 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.711467392Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-02-05T07:51:28.865 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.7117671Z level=info msg="Executing migration" id="alerting notification permissions" 2024-02-05T07:51:28.865 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.712051912Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-02-05T07:51:28.865 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.712422351Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-02-05T07:51:28.865 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.712826861Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-02-05T07:51:28.865 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.713766855Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-02-05T07:51:28.865 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.714057019Z level=info msg="Executing migration" id="create correlation table v1" 2024-02-05T07:51:28.865 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.714430321Z level=info msg="Executing migration" id="add index correlations.uid" 2024-02-05T07:51:28.865 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.714831006Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-02-05T07:51:28.865 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.715245574Z level=info msg="Executing migration" id="add correlation config column" 2024-02-05T07:51:28.866 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.716225382Z level=info msg="Executing migration" id="create entity_events table" 2024-02-05T07:51:28.866 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.716639498Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-02-05T07:51:28.866 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.717088213Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-05T07:51:28.866 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.717179191Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-05T07:51:28.866 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.717463749Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-05T07:51:28.866 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.717564564Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-05T07:51:28.866 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.71781258Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-02-05T07:51:28.866 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.71816937Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-02-05T07:51:28.866 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.718560736Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-02-05T07:51:28.866 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.718983859Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-05T07:51:28.866 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.719385036Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-02-05T07:51:28.866 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.719815798Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-05T07:51:28.866 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.720216692Z level=info msg="Executing migration" id="Drop public config table" 2024-02-05T07:51:28.866 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.720597333Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-02-05T07:51:28.867 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.720999937Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-02-05T07:51:28.867 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.721379913Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-05T07:51:28.867 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.721821588Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-02-05T07:51:28.867 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.722228667Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-02-05T07:51:28.867 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.725195263Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-02-05T07:51:28.867 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.726221623Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-02-05T07:51:28.867 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.727248103Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-02-05T07:51:28.867 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.727570388Z level=info msg="Executing migration" id="add share column" 2024-02-05T07:51:28.867 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.728617634Z level=info msg="Executing migration" id="create default alerting folders" 2024-02-05T07:51:28.867 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.729052487Z level=info msg="Executing migration" id="create file table" 2024-02-05T07:51:28.867 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.7294826Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-02-05T07:51:28.867 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.729925192Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-02-05T07:51:28.867 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.730384565Z level=info msg="Executing migration" id="create file_meta table" 2024-02-05T07:51:28.868 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.730809681Z level=info msg="Executing migration" id="file table idx: path key" 2024-02-05T07:51:28.868 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.73125393Z level=info msg="Executing migration" id="set path collation in file table" 2024-02-05T07:51:28.868 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.731494358Z level=info msg="Executing migration" id="managed permissions migration" 2024-02-05T07:51:28.868 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.731870068Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-02-05T07:51:28.868 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.732131988Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-02-05T07:51:28.868 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.732491103Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-02-05T07:51:28.868 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.733470935Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-02-05T07:51:28.868 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.733775176Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-02-05T07:51:28.868 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.734203995Z level=info msg="Executing migration" id="update group index for alert rules" 2024-02-05T07:51:28.868 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.734560905Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-02-05T07:51:28.868 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.734866732Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-02-05T07:51:28.869 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.735176368Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-02-05T07:51:28.869 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.73620576Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-02-05T07:51:28.869 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.737229426Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-02-05T07:51:28.870 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.737644894Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-02-05T07:51:28.870 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.74738679Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-02-05T07:51:28.870 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.747828274Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-02-05T07:51:28.870 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.748240844Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-02-05T07:51:28.870 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.751658376Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-02-05T07:51:28.870 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.75196077Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-02-05T07:51:28.870 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.752246028Z level=info msg="Executing migration" id="create folder table" 2024-02-05T07:51:28.870 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.752652743Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-02-05T07:51:28.870 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.753091846Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-02-05T07:51:28.871 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.75353699Z level=info msg="Executing migration" id="Update folder title length" 2024-02-05T07:51:28.871 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.753815927Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-02-05T07:51:28.871 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=migrator t=2024-02-05T07:51:28.754233119Z level=info msg="migrations completed" performed=485 skipped=0 duration=243.660062ms 2024-02-05T07:51:28.871 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=sqlstore t=2024-02-05T07:51:28.754628367Z level=info msg="Created default organization" 2024-02-05T07:51:28.871 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=plugin.loader t=2024-02-05T07:51:28.811660667Z level=info msg="Plugin registered" pluginID=input 2024-02-05T07:51:28.871 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=plugin.loader t=2024-02-05T07:51:28.83861625Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-02-05T07:51:28.871 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=plugin.loader t=2024-02-05T07:51:28.838650332Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-02-05T07:51:28.871 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=secrets t=2024-02-05T07:51:28.838687531Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-02-05T07:51:28.871 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=query_data t=2024-02-05T07:51:28.841099481Z level=info msg="Query Service initialization" 2024-02-05T07:51:28.871 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=live.push_http t=2024-02-05T07:51:28.844396175Z level=info msg="Live Push Gateway initialization" 2024-02-05T07:51:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-mon[33345]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:28 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:29.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:28 smithi131 ceph-mon[29507]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:29.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:28 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:29.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:28 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:29.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:28 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:29.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:28 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:28 smithi131 ceph-mon[41654]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:28 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:28 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:28 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:29.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:28 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:29.678 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:29 smithi131 systemd[1]: Starting Ceph node-exporter.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T07:51:29.721 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=infra.usagestats.collector t=2024-02-05T07:51:29.466661244Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-02-05T07:51:29.721 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=provisioning.datasources t=2024-02-05T07:51:29.467677493Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-05T07:51:29.722 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=provisioning.datasources t=2024-02-05T07:51:29.47894056Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-02-05T07:51:29.722 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=provisioning.alerting t=2024-02-05T07:51:29.490787656Z level=info msg="starting to provision alerting" 2024-02-05T07:51:29.722 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=provisioning.alerting t=2024-02-05T07:51:29.490824334Z level=info msg="finished to provision alerting" 2024-02-05T07:51:29.722 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=ngalert.state.manager t=2024-02-05T07:51:29.491035514Z level=info msg="Warming state cache for startup" 2024-02-05T07:51:29.722 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=grafanaStorageLogger t=2024-02-05T07:51:29.491018444Z level=info msg="storage starting" 2024-02-05T07:51:29.722 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=ngalert.state.manager t=2024-02-05T07:51:29.491177849Z level=info msg="State cache has been initialized" states=0 duration=141.879µs 2024-02-05T07:51:29.722 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=ngalert.multiorg.alertmanager t=2024-02-05T07:51:29.491221344Z level=info msg="starting MultiOrg Alertmanager" 2024-02-05T07:51:29.722 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=ticker t=2024-02-05T07:51:29.491380667Z level=info msg=starting first_tick=2024-02-05T07:51:30Z 2024-02-05T07:51:29.722 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=http.server t=2024-02-05T07:51:29.492887553Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-02-05T07:51:29.722 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 07:51:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=sqlstore.transactions t=2024-02-05T07:51:29.511034516Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-05T07:51:30.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:29 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:30.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:29 smithi131 ceph-mon[29507]: Deploying daemon node-exporter.a on smithi131 2024-02-05T07:51:30.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:29 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:30.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:29 smithi131 ceph-mon[41654]: Deploying daemon node-exporter.a on smithi131 2024-02-05T07:51:30.018 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:29 smithi131 bash[80151]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-05T07:51:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:29 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:29 smithi157 ceph-mon[33345]: Deploying daemon node-exporter.a on smithi131 2024-02-05T07:51:30.518 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:30 smithi131 bash[80151]: Getting image source signatures 2024-02-05T07:51:30.518 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:30 smithi131 bash[80151]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-05T07:51:30.518 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:30 smithi131 bash[80151]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-05T07:51:30.518 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:30 smithi131 bash[80151]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-05T07:51:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:30 smithi157 ceph-mon[33345]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:30 smithi131 ceph-mon[41654]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:30 smithi131 ceph-mon[29507]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:31.767 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:31 smithi131 bash[80151]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-05T07:51:32.268 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:31 smithi131 bash[80151]: Writing manifest to image destination 2024-02-05T07:51:32.583 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 podman[80151]: 2024-02-05 07:51:32.216067646 +0000 UTC m=+2.632310508 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-05T07:51:32.583 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 podman[80151]: 2024-02-05 07:51:32.322944521 +0000 UTC m=+2.739187379 container create dfb42e8145217684c22cd354db2f0837d50d6939f1f8fa2bf7ce1961a6bb3895 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:32.878 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 podman[80151]: 2024-02-05 07:51:32.583381099 +0000 UTC m=+2.999623956 container init dfb42e8145217684c22cd354db2f0837d50d6939f1f8fa2bf7ce1961a6bb3895 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:32.878 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 podman[80151]: 2024-02-05 07:51:32.588983717 +0000 UTC m=+3.005226575 container start dfb42e8145217684c22cd354db2f0837d50d6939f1f8fa2bf7ce1961a6bb3895 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:32.878 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.593Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-05T07:51:32.878 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.593Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-05T07:51:32.878 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-05T07:51:32.878 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-05T07:51:32.878 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-05T07:51:32.878 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-05T07:51:32.879 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-05T07:51:32.879 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=arp 2024-02-05T07:51:32.879 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-05T07:51:32.879 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-05T07:51:32.879 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-05T07:51:32.879 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-05T07:51:32.879 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-05T07:51:32.879 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-05T07:51:32.879 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-05T07:51:32.879 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-05T07:51:32.879 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=edac 2024-02-05T07:51:32.879 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-05T07:51:32.879 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-05T07:51:32.879 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-05T07:51:32.879 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-05T07:51:32.880 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-05T07:51:32.880 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-05T07:51:32.880 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-05T07:51:32.880 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-05T07:51:32.880 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-05T07:51:32.880 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-05T07:51:32.880 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-05T07:51:32.880 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-05T07:51:32.880 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-05T07:51:32.880 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-05T07:51:32.880 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-05T07:51:32.880 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-05T07:51:32.880 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=os 2024-02-05T07:51:32.880 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-05T07:51:32.881 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-05T07:51:32.881 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-05T07:51:32.881 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-05T07:51:32.881 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-05T07:51:32.881 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-05T07:51:32.881 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-05T07:51:32.881 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=stat 2024-02-05T07:51:32.881 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-05T07:51:32.881 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-05T07:51:32.881 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-05T07:51:32.881 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=time 2024-02-05T07:51:32.881 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-05T07:51:32.881 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=uname 2024-02-05T07:51:32.881 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-05T07:51:32.881 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-05T07:51:32.882 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.594Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-05T07:51:32.882 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.595Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-05T07:51:32.882 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a[80201]: ts=2024-02-05T07:51:32.595Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-05T07:51:32.882 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 bash[80151]: dfb42e8145217684c22cd354db2f0837d50d6939f1f8fa2bf7ce1961a6bb3895 2024-02-05T07:51:32.882 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 07:51:32 smithi131 systemd[1]: Started Ceph node-exporter.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T07:51:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:32 smithi157 ceph-mon[33345]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:32 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:32 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:32 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:32 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:51:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:51:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-mon[29507]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-mon[41654]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:32 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:33.882 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:33 smithi157 systemd[1]: Starting Ceph node-exporter.b for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T07:51:34.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:33 smithi157 ceph-mon[33345]: Deploying daemon node-exporter.b on smithi157 2024-02-05T07:51:34.221 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:33 smithi157 bash[67437]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-05T07:51:34.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:33 smithi131 ceph-mon[29507]: Deploying daemon node-exporter.b on smithi157 2024-02-05T07:51:34.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:33 smithi131 ceph-mon[41654]: Deploying daemon node-exporter.b on smithi157 2024-02-05T07:51:34.721 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:34 smithi157 bash[67437]: Getting image source signatures 2024-02-05T07:51:34.721 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:34 smithi157 bash[67437]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-05T07:51:34.721 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:34 smithi157 bash[67437]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-05T07:51:34.721 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:34 smithi157 bash[67437]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-05T07:51:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:34 smithi157 ceph-mon[33345]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:34 smithi131 ceph-mon[41654]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:34 smithi131 ceph-mon[29507]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:35.971 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:35 smithi157 bash[67437]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-05T07:51:35.971 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:35 smithi157 bash[67437]: Writing manifest to image destination 2024-02-05T07:51:36.471 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:36 smithi157 podman[67437]: 2024-02-05 07:51:36.098320459 +0000 UTC m=+2.237629479 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-05T07:51:36.471 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:36 smithi157 podman[67437]: 2024-02-05 07:51:36.256027469 +0000 UTC m=+2.395336486 container create 1c3496d1f04ded13db50fecb67d2f38aafff0f17bb63559704ae9a4faf0ed78e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T07:51:37.213 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:36 smithi157 ceph-mon[33345]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:36 smithi131 ceph-mon[41654]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:37.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:36 smithi131 ceph-mon[29507]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:38.471 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:51:38.472 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 podman[67437]: 2024-02-05 07:51:38.168919832 +0000 UTC m=+4.308228856 container init 1c3496d1f04ded13db50fecb67d2f38aafff0f17bb63559704ae9a4faf0ed78e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T07:51:38.472 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 podman[67437]: 2024-02-05 07:51:38.176374352 +0000 UTC m=+4.315683378 container start 1c3496d1f04ded13db50fecb67d2f38aafff0f17bb63559704ae9a4faf0ed78e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T07:51:38.472 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.181Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-05T07:51:38.472 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.181Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-05T07:51:38.472 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.181Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-05T07:51:38.472 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.181Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-05T07:51:38.472 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.181Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-05T07:51:38.473 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-05T07:51:38.473 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-05T07:51:38.473 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=arp 2024-02-05T07:51:38.473 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-05T07:51:38.473 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-05T07:51:38.473 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-05T07:51:38.473 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-05T07:51:38.473 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-05T07:51:38.473 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-05T07:51:38.473 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-05T07:51:38.473 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-05T07:51:38.473 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=edac 2024-02-05T07:51:38.473 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-05T07:51:38.473 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-05T07:51:38.473 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-05T07:51:38.474 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-05T07:51:38.474 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-05T07:51:38.474 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-05T07:51:38.474 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-05T07:51:38.474 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-05T07:51:38.474 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-05T07:51:38.474 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-05T07:51:38.474 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-05T07:51:38.474 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-05T07:51:38.474 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-05T07:51:38.474 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-05T07:51:38.474 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-05T07:51:38.474 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-05T07:51:38.474 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=os 2024-02-05T07:51:38.474 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-05T07:51:38.475 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-05T07:51:38.475 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-05T07:51:38.475 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-05T07:51:38.475 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-05T07:51:38.475 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-05T07:51:38.475 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-05T07:51:38.475 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=stat 2024-02-05T07:51:38.475 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-05T07:51:38.475 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-05T07:51:38.475 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-05T07:51:38.475 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=time 2024-02-05T07:51:38.475 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-05T07:51:38.475 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=uname 2024-02-05T07:51:38.475 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-05T07:51:38.476 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-05T07:51:38.476 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-05T07:51:38.476 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-05T07:51:38.476 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b[67488]: ts=2024-02-05T07:51:38.182Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-05T07:51:39.166 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:38 smithi157 ceph-mon[33345]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:38 smithi131 ceph-mon[41654]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:39.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:38 smithi131 ceph-mon[29507]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:39.471 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:39 smithi157 bash[67437]: 1c3496d1f04ded13db50fecb67d2f38aafff0f17bb63559704ae9a4faf0ed78e 2024-02-05T07:51:39.471 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 07:51:39 smithi157 systemd[1]: Started Ceph node-exporter.b for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T07:51:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:39 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:39 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:39 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:39 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:39 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:39 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:39 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:51:40.247 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:39 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:40.247 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:39 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:40.247 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:39 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:40.247 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:39 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:40.247 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:39 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:40.247 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:39 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:40.247 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:39 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:51:40.248 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:39 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:40.248 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:39 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:40.248 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:39 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:40.248 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:39 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:40.248 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:39 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:40.248 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:39 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:40.248 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:39 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:51:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:40 smithi157 ceph-mon[33345]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:40 smithi131 ceph-mon[41654]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:41.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:40 smithi131 ceph-mon[29507]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:42 smithi157 ceph-mon[33345]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:42 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:42 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:42 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:42 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:43.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:51:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:51:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:42 smithi131 ceph-mon[41654]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:42 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:42 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:42 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:42 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:42 smithi131 ceph-mon[29507]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:42 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:42 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:42 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:42 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:44.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:44 smithi157 ceph-mon[33345]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:44 smithi131 ceph-mon[41654]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:45.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:44 smithi131 ceph-mon[29507]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:46.865 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:46 smithi131 ceph-mon[41654]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:51:46.865 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:46 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:46.865 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:46 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:46.865 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:46 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:51:46.865 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:46 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:51:46.865 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:46 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:46.865 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:46 smithi131 ceph-mon[29507]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:51:46.866 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:46 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:46.866 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:46 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:46.866 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:46 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:51:46.866 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:46 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:51:46.866 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:46 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:46.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:46 smithi157 ceph-mon[33345]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:51:46.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:46 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:46.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:46 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:46.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:46 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:51:46.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:46 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:51:46.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:46 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:47.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:47 smithi131 ceph-mon[29507]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-05T07:51:47.756 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:47 smithi131 ceph-mon[29507]: Reconfiguring daemon alertmanager.a on smithi131 2024-02-05T07:51:47.757 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:47 smithi131 ceph-mon[41654]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-05T07:51:47.757 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:47 smithi131 ceph-mon[41654]: Reconfiguring daemon alertmanager.a on smithi131 2024-02-05T07:51:47.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:47 smithi157 ceph-mon[33345]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-05T07:51:47.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:47 smithi157 ceph-mon[33345]: Reconfiguring daemon alertmanager.a on smithi131 2024-02-05T07:51:48.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:51:48 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:51:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:48 smithi157 ceph-mon[33345]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:51:49.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:48 smithi131 ceph-mon[41654]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:51:49.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:48 smithi131 ceph-mon[29507]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:51:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:49 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:50.018 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:49 smithi131 systemd[1]: Stopping Ceph alertmanager.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T07:51:50.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:49 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:49 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:50.346 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:50 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[79716]: ts=2024-02-05T07:51:50.081Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-05T07:51:50.346 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:50 smithi131 podman[81024]: 2024-02-05 07:51:50.101920666 +0000 UTC m=+0.162116796 container died 19d0053734f7c786a34a489dfe73809d323fa5533b37a2022dcea56e1846039e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:50.768 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:50 smithi131 podman[81024]: 2024-02-05 07:51:50.553310011 +0000 UTC m=+0.613506180 container cleanup 19d0053734f7c786a34a489dfe73809d323fa5533b37a2022dcea56e1846039e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:50.768 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:50 smithi131 bash[81024]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a 2024-02-05T07:51:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:50 smithi157 ceph-mon[33345]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:51.233 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:50 smithi131 ceph-mon[41654]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:51.233 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:50 smithi131 ceph-mon[29507]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:51.234 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:50 smithi131 podman[81043]: 2024-02-05 07:51:50.821642464 +0000 UTC m=+0.715047264 container remove 19d0053734f7c786a34a489dfe73809d323fa5533b37a2022dcea56e1846039e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:51.234 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:50 smithi131 podman[81043]: 2024-02-05 07:51:50.888973515 +0000 UTC m=+0.782378315 volume remove 0ac4d9b2e8af786708da7c15327dd294b475d1ea0074219e0af800b85d807cdd 2024-02-05T07:51:51.234 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-05T07:51:51.234 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 systemd[1]: Stopped Ceph alertmanager.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T07:51:51.234 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 systemd[1]: Starting Ceph alertmanager.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T07:51:51.505 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 podman[81139]: 2024-02-05 07:51:51.136500881 +0000 UTC m=+0.018083490 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-02-05T07:51:51.505 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 podman[81139]: 2024-02-05 07:51:51.252198124 +0000 UTC m=+0.133780731 volume create 87e0ef917bc7979f9d8b271e17dae631254acb5ca824f673b0b26ca06dbc5e86 2024-02-05T07:51:51.505 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 podman[81139]: 2024-02-05 07:51:51.323834211 +0000 UTC m=+0.205416821 container create 6506f3a1485ab069cb8edd5e1c136c7abf6b71a4735388d6f43126040590e5e7 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:52.018 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 podman[81139]: 2024-02-05 07:51:51.608532638 +0000 UTC m=+0.490115249 container init 6506f3a1485ab069cb8edd5e1c136c7abf6b71a4735388d6f43126040590e5e7 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:52.018 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 podman[81139]: 2024-02-05 07:51:51.613811531 +0000 UTC m=+0.495394140 container start 6506f3a1485ab069cb8edd5e1c136c7abf6b71a4735388d6f43126040590e5e7 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:52.018 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[81153]: ts=2024-02-05T07:51:51.625Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-05T07:51:52.018 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[81153]: ts=2024-02-05T07:51:51.625Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-05T07:51:52.018 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[81153]: ts=2024-02-05T07:51:51.626Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.131 port=9094 2024-02-05T07:51:52.018 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[81153]: ts=2024-02-05T07:51:51.627Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-05T07:51:52.018 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[81153]: ts=2024-02-05T07:51:51.650Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-05T07:51:52.018 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[81153]: ts=2024-02-05T07:51:51.651Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-05T07:51:52.019 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[81153]: ts=2024-02-05T07:51:51.654Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-05T07:51:52.019 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[81153]: ts=2024-02-05T07:51:51.654Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-05T07:51:52.019 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 bash[81139]: 6506f3a1485ab069cb8edd5e1c136c7abf6b71a4735388d6f43126040590e5e7 2024-02-05T07:51:52.019 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:51 smithi131 systemd[1]: Started Ceph alertmanager.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T07:51:52.805 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:52 smithi157 ceph-mon[33345]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:51:52.805 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:52 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:52.805 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:52 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:52.805 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:52 smithi157 ceph-mon[33345]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-05T07:51:53.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:51:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:51:53.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:52 smithi131 ceph-mon[41654]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:51:53.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:52 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:53.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:52 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:53.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:52 smithi131 ceph-mon[41654]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-05T07:51:53.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:52 smithi131 ceph-mon[29507]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:51:53.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:52 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:53.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:52 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:53.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:52 smithi131 ceph-mon[29507]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-05T07:51:53.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:53 smithi157 ceph-mon[33345]: Reconfiguring daemon prometheus.a on smithi157 2024-02-05T07:51:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:53 smithi131 ceph-mon[41654]: Reconfiguring daemon prometheus.a on smithi157 2024-02-05T07:51:54.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:53 smithi131 ceph-mon[29507]: Reconfiguring daemon prometheus.a on smithi157 2024-02-05T07:51:54.018 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:51:53 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[81153]: ts=2024-02-05T07:51:53.628Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.00056523s 2024-02-05T07:51:55.096 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:54 smithi157 ceph-mon[33345]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:55.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:54 smithi131 ceph-mon[41654]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:55.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:54 smithi131 ceph-mon[29507]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:55.471 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:55 smithi157 systemd[1]: Stopping Ceph prometheus.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T07:51:55.827 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:51:55.504Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-05T07:51:55.827 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:51:55.505Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-05T07:51:55.828 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:51:55.505Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-05T07:51:55.828 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:51:55.505Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-05T07:51:55.828 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:51:55.505Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-05T07:51:55.828 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:51:55.505Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-05T07:51:55.828 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:51:55.505Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-05T07:51:55.828 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:51:55.505Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-05T07:51:55.828 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:51:55.505Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-05T07:51:55.828 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:51:55.505Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-05T07:51:55.828 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:51:55.505Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-05T07:51:55.828 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[65143]: ts=2024-02-05T07:51:55.505Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-05T07:51:55.828 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:55 smithi157 podman[68529]: 2024-02-05 07:51:55.515143379 +0000 UTC m=+0.136397338 container died 6e137e49b25a93ee24668bfa603aed1e89769ee73e81a15ef43442d8989983c4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:56.143 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:55 smithi157 podman[68529]: 2024-02-05 07:51:55.942056091 +0000 UTC m=+0.563310054 container cleanup 6e137e49b25a93ee24668bfa603aed1e89769ee73e81a15ef43442d8989983c4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:56.143 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:55 smithi157 bash[68529]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a 2024-02-05T07:51:56.471 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 podman[68548]: 2024-02-05 07:51:56.143206142 +0000 UTC m=+0.629939305 container remove 6e137e49b25a93ee24668bfa603aed1e89769ee73e81a15ef43442d8989983c4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:56.471 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-02-05T07:51:56.471 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 systemd[1]: Stopped Ceph prometheus.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T07:51:56.471 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@prometheus.a.service: Consumed 1.151s CPU time. 2024-02-05T07:51:56.471 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 systemd[1]: Starting Ceph prometheus.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T07:51:56.846 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 podman[68648]: 2024-02-05 07:51:56.469770145 +0000 UTC m=+0.019184948 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-05T07:51:56.846 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 podman[68648]: 2024-02-05 07:51:56.585824525 +0000 UTC m=+0.135239325 container create 122617c83a3d69e5b6dd8b6abffe710a22c966f0fb0c7c582b9420e4ba494357 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:57.154 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:56 smithi131 ceph-mon[41654]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:57.154 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:56 smithi131 ceph-mon[29507]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:57.155 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE Bus STOPPING 2024-02-05T07:51:57.155 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-05T07:51:57.155 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE Bus STOPPED 2024-02-05T07:51:57.155 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE Bus STARTING 2024-02-05T07:51:57.155 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE Serving on http://:::9283 2024-02-05T07:51:57.155 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE Bus STARTED 2024-02-05T07:51:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-mon[33345]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:57.221 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 podman[68648]: 2024-02-05 07:51:56.846297344 +0000 UTC m=+0.395712149 container init 122617c83a3d69e5b6dd8b6abffe710a22c966f0fb0c7c582b9420e4ba494357 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:57.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 podman[68648]: 2024-02-05 07:51:56.851772032 +0000 UTC m=+0.401186834 container start 122617c83a3d69e5b6dd8b6abffe710a22c966f0fb0c7c582b9420e4ba494357 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:51:57.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.876Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-05T07:51:57.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.876Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-05T07:51:57.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.876Z caller=main.go:570 level=info host_details="(Linux 5.14.0-416.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Jan 31 22:52:48 UTC 2024 x86_64 smithi157 (none))" 2024-02-05T07:51:57.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.876Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-05T07:51:57.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.876Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-05T07:51:57.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.878Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-05T07:51:57.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.878Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-05T07:51:57.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.879Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-05T07:51:57.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.879Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-05T07:51:57.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.881Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-05T07:51:57.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.881Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=3.189µs 2024-02-05T07:51:57.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.881Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-05T07:51:57.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.882Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-02-05T07:51:57.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.882Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-02-05T07:51:57.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.882Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=34.002µs wal_replay_duration=1.132361ms wbl_replay_duration=107ns total_replay_duration=1.246126ms 2024-02-05T07:51:57.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.883Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-05T07:51:57.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.883Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-05T07:51:57.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.883Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-05T07:51:57.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.901Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=17.73882ms db_storage=778ns remote_storage=1.05µs web_handler=362ns query_engine=588ns scrape=395.036µs scrape_sd=75.72µs notify=9.7µs notify_sd=11.606µs rules=17.043068ms tracing=6.265µs 2024-02-05T07:51:57.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.901Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-05T07:51:57.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T07:51:56.901Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-05T07:51:57.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 bash[68648]: 122617c83a3d69e5b6dd8b6abffe710a22c966f0fb0c7c582b9420e4ba494357 2024-02-05T07:51:57.223 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 07:51:56 smithi157 systemd[1]: Started Ceph prometheus.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T07:51:57.518 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE Bus STOPPING 2024-02-05T07:51:57.518 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-05T07:51:57.518 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE Bus STOPPED 2024-02-05T07:51:57.518 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE Bus STARTING 2024-02-05T07:51:57.518 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE Serving on http://:::9283 2024-02-05T07:51:57.518 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE Bus STARTED 2024-02-05T07:51:57.519 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE Bus STOPPING 2024-02-05T07:51:57.519 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-05T07:51:57.519 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE Bus STOPPED 2024-02-05T07:51:57.519 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE Bus STARTING 2024-02-05T07:51:57.519 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE Serving on http://:::9283 2024-02-05T07:51:57.519 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:07:51:57] ENGINE Bus STARTED 2024-02-05T07:51:58.188 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:58.188 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:58.188 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-05T07:51:58.188 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-05T07:51:58.189 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-02-05T07:51:58.189 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-02-05T07:51:58.189 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:58.189 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-05T07:51:58.189 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-05T07:51:58.189 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi157.front.sepia.ceph.com:9095"}]: dispatch 2024-02-05T07:51:58.189 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi157.front.sepia.ceph.com:9095"}]: dispatch 2024-02-05T07:51:58.189 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:58.189 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-05T07:51:58.189 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-05T07:51:58.189 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi157.front.sepia.ceph.com:3000"}]: dispatch 2024-02-05T07:51:58.189 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi157.front.sepia.ceph.com:3000"}]: dispatch 2024-02-05T07:51:58.189 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:58.189 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:51:58.189 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:58.190 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:58.190 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:58.190 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:58.190 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-05T07:51:58.190 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-05T07:51:58.190 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-02-05T07:51:58.190 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-02-05T07:51:58.190 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:58.190 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-05T07:51:58.190 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-05T07:51:58.190 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi157.front.sepia.ceph.com:9095"}]: dispatch 2024-02-05T07:51:58.190 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi157.front.sepia.ceph.com:9095"}]: dispatch 2024-02-05T07:51:58.191 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:58.191 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-05T07:51:58.191 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-05T07:51:58.191 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi157.front.sepia.ceph.com:3000"}]: dispatch 2024-02-05T07:51:58.191 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi157.front.sepia.ceph.com:3000"}]: dispatch 2024-02-05T07:51:58.191 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:58.191 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:51:58.191 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:58.191 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:57 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-05T07:51:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-05T07:51:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-02-05T07:51:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-02-05T07:51:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-05T07:51:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-05T07:51:58.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi157.front.sepia.ceph.com:9095"}]: dispatch 2024-02-05T07:51:58.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi157.front.sepia.ceph.com:9095"}]: dispatch 2024-02-05T07:51:58.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:58.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-05T07:51:58.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-05T07:51:58.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi157.front.sepia.ceph.com:3000"}]: dispatch 2024-02-05T07:51:58.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi157.front.sepia.ceph.com:3000"}]: dispatch 2024-02-05T07:51:58.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:51:58.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:51:58.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:58.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:57 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:58.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:51:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:51:59.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:51:59 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:59.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:51:59 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:51:59 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:00.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:00 smithi131 ceph-mon[41654]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:00 smithi131 ceph-mon[29507]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:00.393 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:00 smithi157 ceph-mon[33345]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:01.631 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:01 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:52:01.631 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:01 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:52:01.632 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:01 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:52:01.632 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:01 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:52:01.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:01 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:52:01.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:01 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:52:02.018 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 07:52:01 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[81153]: ts=2024-02-05T07:52:01.630Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003278635s 2024-02-05T07:52:02.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:02 smithi131 ceph-mon[41654]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:02.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:02 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:52:02.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:02 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:52:02.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:02 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:52:02.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:02 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:52:02.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:02 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:52:02.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:02 smithi131 ceph-mon[29507]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:02.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:02 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:52:02.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:02 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:52:02.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:02 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:52:02.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:02 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:52:02.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:02 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:52:02.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:02 smithi157 ceph-mon[33345]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:02.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:02 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:52:02.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:02 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:52:02.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:02 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:52:02.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:02 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:52:02.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:02 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:52:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:52:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:52:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:52:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:04 smithi157 ceph-mon[33345]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:05.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:04 smithi131 ceph-mon[29507]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:05.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:04 smithi131 ceph-mon[41654]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:06 smithi157 ceph-mon[33345]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:07.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:06 smithi131 ceph-mon[41654]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:06 smithi131 ceph-mon[29507]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:08.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:52:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:52:09.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:08 smithi157 ceph-mon[33345]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:09.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:08 smithi131 ceph-mon[41654]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:09.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:08 smithi131 ceph-mon[29507]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:09 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:10.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:09 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:09 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:10 smithi157 ceph-mon[33345]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:10 smithi131 ceph-mon[41654]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:11.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:10 smithi131 ceph-mon[29507]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:12 smithi157 ceph-mon[33345]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:12 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:13.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:52:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:52:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:52:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:12 smithi131 ceph-mon[29507]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:12 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:12 smithi131 ceph-mon[41654]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:12 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:14 smithi157 ceph-mon[33345]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:15.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:14 smithi131 ceph-mon[41654]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:14 smithi131 ceph-mon[29507]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:16 smithi157 ceph-mon[33345]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:16 smithi131 ceph-mon[41654]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:17.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:16 smithi131 ceph-mon[29507]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:18.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:52:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:52:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:18 smithi157 ceph-mon[33345]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:19.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:18 smithi131 ceph-mon[41654]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:18 smithi131 ceph-mon[29507]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:19 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:20.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:19 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:20.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:19 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:20 smithi157 ceph-mon[33345]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:20 smithi131 ceph-mon[41654]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:21.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:20 smithi131 ceph-mon[29507]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:22 smithi157 ceph-mon[33345]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:22 smithi131 ceph-mon[41654]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:23.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:52:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:52:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:52:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:22 smithi131 ceph-mon[29507]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:24 smithi157 ceph-mon[33345]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:25.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:24 smithi131 ceph-mon[41654]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:25.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:24 smithi131 ceph-mon[29507]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:26 smithi157 ceph-mon[33345]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:26 smithi131 ceph-mon[29507]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:26 smithi131 ceph-mon[41654]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:28.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:28.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:28.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:52:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:52:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:28 smithi157 ceph-mon[33345]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:28 smithi131 ceph-mon[41654]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:29.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:28 smithi131 ceph-mon[29507]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:29 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:29 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:30.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:29 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:30 smithi157 ceph-mon[33345]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:30 smithi131 ceph-mon[41654]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:30 smithi131 ceph-mon[29507]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:32 smithi157 ceph-mon[33345]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:32 smithi131 ceph-mon[41654]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:32 smithi131 ceph-mon[29507]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:33.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:52:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:52:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:52:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:34 smithi157 ceph-mon[33345]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:34 smithi131 ceph-mon[41654]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:34 smithi131 ceph-mon[29507]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:36 smithi157 ceph-mon[33345]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:36 smithi131 ceph-mon[41654]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:37.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:36 smithi131 ceph-mon[29507]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:37.592 INFO:tasks.workunit.client.0.smithi131.stderr:Note: switching to '164d3be4a527be775f4d3c343ea591122f1b8921'. 2024-02-05T07:52:37.592 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-02-05T07:52:37.592 INFO:tasks.workunit.client.0.smithi131.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-05T07:52:37.592 INFO:tasks.workunit.client.0.smithi131.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-05T07:52:37.592 INFO:tasks.workunit.client.0.smithi131.stderr:state without impacting any branches by switching back to a branch. 2024-02-05T07:52:37.593 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-02-05T07:52:37.593 INFO:tasks.workunit.client.0.smithi131.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-05T07:52:37.593 INFO:tasks.workunit.client.0.smithi131.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-05T07:52:37.593 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-02-05T07:52:37.593 INFO:tasks.workunit.client.0.smithi131.stderr: git switch -c 2024-02-05T07:52:37.593 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-02-05T07:52:37.593 INFO:tasks.workunit.client.0.smithi131.stderr:Or undo this operation with: 2024-02-05T07:52:37.593 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-02-05T07:52:37.593 INFO:tasks.workunit.client.0.smithi131.stderr: git switch - 2024-02-05T07:52:37.593 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-02-05T07:52:37.593 INFO:tasks.workunit.client.0.smithi131.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-05T07:52:37.593 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-02-05T07:52:37.593 INFO:tasks.workunit.client.0.smithi131.stderr:HEAD is now at 164d3be4a52 Merge branch 'subclass-thread' of https://github.com/guits/ceph into wip-guits-testing-3-2024-02-04-2254 2024-02-05T07:52:37.595 DEBUG:teuthology.orchestra.run.smithi131:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-05T07:52:37.626 INFO:tasks.workunit.client.0.smithi131.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-05T07:52:37.628 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-05T07:52:37.628 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-05T07:52:38.384 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-05T07:52:38.461 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-05T07:52:38.501 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-05T07:52:38.502 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-05T07:52:38.503 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-05T07:52:38.542 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-05T07:52:38.545 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-02-05T07:52:38.545 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-05T07:52:38.601 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-02-05T07:52:38.603 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-02-05T07:52:38.603 DEBUG:teuthology.orchestra.run.smithi131:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=164d3be4a527be775f4d3c343ea591122f1b8921 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-02-05T07:52:38.670 INFO:tasks.workunit.client.0.smithi131.stderr:+ parallel=1 2024-02-05T07:52:38.671 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' '' = --serial ']' 2024-02-05T07:52:38.671 INFO:tasks.workunit.client.0.smithi131.stderr:+ crimson=0 2024-02-05T07:52:38.671 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' '' = --crimson ']' 2024-02-05T07:52:38.671 INFO:tasks.workunit.client.0.smithi131.stderr:+ color= 2024-02-05T07:52:38.671 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -t 1 ']' 2024-02-05T07:52:38.671 INFO:tasks.workunit.client.0.smithi131.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-02-05T07:52:38.671 INFO:tasks.workunit.client.0.smithi131.stderr:+ GTEST_OUTPUT=xml:/home/ubuntu/cephtest/archive/gtest_xml_report 2024-02-05T07:52:38.671 INFO:tasks.workunit.client.0.smithi131.stderr:+ declare -A pids 2024-02-05T07:52:38.671 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.671 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.671 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_aio 2024-02-05T07:52:38.671 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_aio' 2024-02-05T07:52:38.671 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_aio 2024-02-05T07:52:38.671 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.672 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_aio 2024-02-05T07:52:38.672 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=81967 2024-02-05T07:52:38.672 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_aio on pid 81967' 2024-02-05T07:52:38.672 INFO:tasks.workunit.client.0.smithi131.stdout:test api_aio on pid 81967 2024-02-05T07:52:38.675 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=81967 2024-02-05T07:52:38.675 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.675 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-02-05T07:52:38.675 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.676 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_aio_pp 2024-02-05T07:52:38.676 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_aio_pp' 2024-02-05T07:52:38.676 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_aio_pp 2024-02-05T07:52:38.676 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.676 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.676 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.676 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.676 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.676 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.676 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.676 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.676 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.676 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.676 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.677 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.677 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.677 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.677 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.677 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.677 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.677 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.677 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.677 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.677 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.677 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.677 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.677 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.677 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.677 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.677 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_aio_pp 2024-02-05T07:52:38.678 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=81974 2024-02-05T07:52:38.678 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_aio_pp on pid 81974' 2024-02-05T07:52:38.678 INFO:tasks.workunit.client.0.smithi131.stdout:test api_aio_pp on pid 81974 2024-02-05T07:52:38.682 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=81974 2024-02-05T07:52:38.682 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.682 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.682 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-02-05T07:52:38.682 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_io 2024-02-05T07:52:38.682 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_io' 2024-02-05T07:52:38.682 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_io 2024-02-05T07:52:38.682 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.682 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.682 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.683 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.683 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.683 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.683 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.683 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.683 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.683 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.683 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.683 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.683 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.683 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.683 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.683 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.683 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.683 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.684 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.684 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.684 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.684 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.684 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.684 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.684 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.684 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.684 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.684 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.684 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.684 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.684 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.684 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.685 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.685 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.685 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.685 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.685 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.685 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.685 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.685 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.685 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.685 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.685 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.686 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.686 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.686 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.686 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_io 2024-02-05T07:52:38.686 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=81983 2024-02-05T07:52:38.687 INFO:tasks.workunit.client.0.smithi131.stdout:test api_io on pid 81983 2024-02-05T07:52:38.691 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_io on pid 81983' 2024-02-05T07:52:38.691 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=81983 2024-02-05T07:52:38.691 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.691 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.691 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-02-05T07:52:38.691 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_io_pp 2024-02-05T07:52:38.691 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_io_pp' 2024-02-05T07:52:38.691 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_io_pp 2024-02-05T07:52:38.691 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.691 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.692 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.692 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.692 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.692 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.692 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.692 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.692 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.692 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.692 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.693 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.693 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.693 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.693 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.693 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.693 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.693 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.693 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.693 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.693 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.693 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.693 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.693 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.693 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.694 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.694 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.694 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.694 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.694 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.694 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.694 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.694 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.694 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.694 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.694 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.694 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.694 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.694 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.694 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.694 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.695 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.695 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.695 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.695 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.695 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.695 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.695 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.695 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.695 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.695 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.695 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.695 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.695 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.695 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.695 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_io_pp 2024-02-05T07:52:38.696 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=81994 2024-02-05T07:52:38.696 INFO:tasks.workunit.client.0.smithi131.stdout:test api_io_pp on pid 81994 2024-02-05T07:52:38.700 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_io_pp on pid 81994' 2024-02-05T07:52:38.700 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=81994 2024-02-05T07:52:38.700 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.701 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-02-05T07:52:38.701 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.701 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_asio 2024-02-05T07:52:38.701 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_asio' 2024-02-05T07:52:38.701 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_asio 2024-02-05T07:52:38.701 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.701 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.701 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.701 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.701 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.701 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.701 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.701 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.701 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.702 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.702 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.702 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.702 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.702 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.702 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.702 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.702 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.702 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.702 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.702 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.702 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.702 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.702 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.702 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.703 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.703 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.703 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.703 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.703 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.703 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.703 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.703 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.703 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.703 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.703 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.703 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.703 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.703 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.703 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.704 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.704 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.704 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.704 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.704 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.704 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.704 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.704 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.704 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.704 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.704 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.704 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.704 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.704 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.704 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.705 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.705 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.705 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.705 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.705 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.705 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.705 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.705 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.705 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.705 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.705 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.705 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.705 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.705 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.705 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.705 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.706 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.706 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.706 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.706 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_asio 2024-02-05T07:52:38.706 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82008 2024-02-05T07:52:38.706 INFO:tasks.workunit.client.0.smithi131.stdout:test api_asio on pid 82008 2024-02-05T07:52:38.713 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_asio on pid 82008' 2024-02-05T07:52:38.713 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82008 2024-02-05T07:52:38.713 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.713 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-02-05T07:52:38.713 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.713 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_list 2024-02-05T07:52:38.713 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.713 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.713 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.713 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.713 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.713 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_list' 2024-02-05T07:52:38.714 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.714 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_list 2024-02-05T07:52:38.714 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.714 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.714 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.714 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.714 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.714 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.714 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.714 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.714 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.714 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.714 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.714 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.714 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.715 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.715 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.715 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.715 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.715 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.715 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.715 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.715 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.715 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.715 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.715 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.715 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.715 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.715 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.716 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.716 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.716 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.716 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.716 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.716 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.716 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.716 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.716 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.716 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.716 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.716 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.716 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.716 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.716 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.717 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.717 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.717 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.717 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.717 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.717 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.717 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.717 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.717 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.717 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.717 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.717 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.717 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.717 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.717 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.717 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.718 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.718 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.718 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.718 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.718 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.718 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.718 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.718 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.718 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.718 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.718 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.718 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.718 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.718 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.718 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.719 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.719 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.719 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.719 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.719 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.719 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.719 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.719 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.719 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.719 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.719 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.719 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.719 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.719 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.719 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.720 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.720 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.720 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.720 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.720 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.720 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.720 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/81967/exe 2024-02-05T07:52:38.720 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_list 2024-02-05T07:52:38.720 INFO:tasks.workunit.client.0.smithi131.stdout:test api_list on pid 82026 2024-02-05T07:52:38.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:52:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:52:38.727 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82026 2024-02-05T07:52:38.727 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_list on pid 82026' 2024-02-05T07:52:38.727 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82026 2024-02-05T07:52:38.727 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.727 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.727 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-02-05T07:52:38.727 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_lock 2024-02-05T07:52:38.727 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_lock' 2024-02-05T07:52:38.727 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.727 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.728 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.728 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.728 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.728 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.728 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.728 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.728 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_lock 2024-02-05T07:52:38.728 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.728 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.728 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.728 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.728 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.728 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.729 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.729 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.729 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.729 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.729 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.729 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.729 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.729 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.729 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.729 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.729 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.729 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.729 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.729 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.729 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.730 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.730 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.730 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.730 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.730 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.730 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.730 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.730 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.730 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.730 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.730 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.730 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.730 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.730 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.730 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.730 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.731 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.731 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.731 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.731 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.731 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.731 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.731 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.731 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.731 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.731 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.731 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.731 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.731 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.731 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.731 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.732 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.732 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.732 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.732 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.732 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.732 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.732 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.732 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.732 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.732 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.732 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.732 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.732 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.732 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.732 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.732 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.733 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.733 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.733 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.733 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_aio.log 2024-02-05T07:52:38.733 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.733 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.733 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.733 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.733 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_aio: /' 2024-02-05T07:52:38.733 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.733 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.733 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.733 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.733 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.733 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.734 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.734 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.734 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.734 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.734 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.734 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.734 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.734 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.734 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.734 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.734 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.734 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.734 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.734 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.734 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.735 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.735 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.735 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.735 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.735 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.735 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.735 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.735 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.735 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.735 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.735 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.735 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.735 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.735 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/81974/exe 2024-02-05T07:52:38.735 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio.xml 2024-02-05T07:52:38.736 INFO:tasks.workunit.client.0.smithi131.stdout:test api_lock on pid 82048 2024-02-05T07:52:38.739 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_lock 2024-02-05T07:52:38.739 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82048 2024-02-05T07:52:38.739 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_lock on pid 82048' 2024-02-05T07:52:38.739 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82048 2024-02-05T07:52:38.739 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.739 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.739 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-02-05T07:52:38.739 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_lock_pp 2024-02-05T07:52:38.739 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.740 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_lock_pp' 2024-02-05T07:52:38.740 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.740 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.740 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.740 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.740 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.740 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.740 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.740 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.740 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.740 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.740 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.740 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.740 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.740 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.741 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.741 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.741 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.741 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.741 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.741 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.741 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.741 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.741 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.741 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.741 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.741 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.741 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.741 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.741 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.742 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.742 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.742 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.742 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.742 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.742 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.742 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.742 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.742 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.742 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.742 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.742 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.742 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.742 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.742 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.742 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.743 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.743 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.743 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.743 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_lock_pp 2024-02-05T07:52:38.743 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.743 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.743 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.743 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.743 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.743 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.743 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.743 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.743 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.743 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.743 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.743 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.744 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.744 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.744 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.744 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.744 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.744 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.744 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.744 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.744 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.744 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.744 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.744 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.744 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.744 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.744 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.745 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.745 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.745 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.745 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.745 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.745 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.745 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/81983/exe 2024-02-05T07:52:38.745 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.745 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.745 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.745 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_aio_pp: /' 2024-02-05T07:52:38.745 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.745 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.746 INFO:tasks.workunit.client.0.smithi131.stdout:test api_lock_pp on pid 82066 2024-02-05T07:52:38.749 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.749 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.749 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.749 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.749 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.749 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.749 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.749 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio_pp.xml 2024-02-05T07:52:38.749 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.749 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.749 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.749 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.749 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.749 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.750 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.750 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.750 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.750 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.750 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_lock_pp 2024-02-05T07:52:38.750 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.750 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.750 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82066 2024-02-05T07:52:38.750 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_lock_pp on pid 82066' 2024-02-05T07:52:38.750 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82066 2024-02-05T07:52:38.750 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-02-05T07:52:38.750 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.750 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.750 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.750 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.751 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.751 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.751 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.751 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.751 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.751 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.751 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.751 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.751 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.751 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_misc 2024-02-05T07:52:38.751 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.751 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.751 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.751 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_misc' 2024-02-05T07:52:38.752 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.752 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.752 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.752 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.752 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.752 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-02-05T07:52:38.752 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.752 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.752 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.752 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.752 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.752 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.752 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.752 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.753 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.753 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.753 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.753 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.753 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.753 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.753 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_io: /' 2024-02-05T07:52:38.753 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.753 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.753 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.753 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.753 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.753 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.753 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.753 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.753 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.754 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.754 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.754 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.754 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.754 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.754 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.754 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_io.log 2024-02-05T07:52:38.754 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.754 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.754 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.754 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.754 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.754 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.754 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.755 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.755 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.755 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.755 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.755 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.755 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.755 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.755 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.755 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.755 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.755 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.755 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.755 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.755 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.755 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.755 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.756 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.756 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.756 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.756 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.756 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.756 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_misc 2024-02-05T07:52:38.756 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.756 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.756 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.756 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.756 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.756 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.756 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.756 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.756 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.757 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.757 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.757 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.757 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.757 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.757 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io.xml 2024-02-05T07:52:38.757 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.757 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.757 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.757 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.757 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.757 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.757 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.757 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.757 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.757 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.758 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.758 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.758 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/81994/exe 2024-02-05T07:52:38.758 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.758 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.758 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.758 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.758 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.758 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.758 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.758 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.758 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.758 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.758 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.758 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.759 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.759 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.759 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.759 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.759 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.759 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.759 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.759 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.759 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.759 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.759 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.759 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.759 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.759 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.759 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.759 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.760 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.760 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.760 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.760 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.760 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.760 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.760 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.760 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.760 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_misc 2024-02-05T07:52:38.760 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82008/exe 2024-02-05T07:52:38.760 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.760 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82090 2024-02-05T07:52:38.760 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.761 INFO:tasks.workunit.client.0.smithi131.stdout:test api_misc on pid 82090 2024-02-05T07:52:38.765 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_misc on pid 82090' 2024-02-05T07:52:38.765 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82090 2024-02-05T07:52:38.765 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.765 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.765 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.765 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.765 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.765 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.765 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.765 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.765 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.765 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.765 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-02-05T07:52:38.766 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.766 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.766 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.766 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.766 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.766 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-02-05T07:52:38.766 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.766 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.766 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.766 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.766 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.766 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.766 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.766 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.766 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.767 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.767 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.767 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.767 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_io_pp: /' 2024-02-05T07:52:38.767 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.767 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.767 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.767 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.767 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.767 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.767 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.767 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.767 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.767 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.767 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.768 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.768 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.768 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.768 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.768 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.768 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.768 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.768 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io_pp.xml 2024-02-05T07:52:38.768 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.768 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.768 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.768 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.768 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.768 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.769 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.769 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.769 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.769 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_asio.log 2024-02-05T07:52:38.769 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_misc_pp 2024-02-05T07:52:38.769 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_misc_pp' 2024-02-05T07:52:38.769 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_asio: /' 2024-02-05T07:52:38.769 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.769 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.769 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.769 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.769 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.769 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_misc_pp 2024-02-05T07:52:38.769 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.769 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.769 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.770 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_asio.xml 2024-02-05T07:52:38.770 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.770 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.770 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.770 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.770 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.770 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.770 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.770 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.770 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.770 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.770 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.770 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.770 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.770 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.771 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.771 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.771 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.771 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.771 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.771 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.771 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.771 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.771 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.771 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.771 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.771 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.771 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.771 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.771 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.771 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.772 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.772 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.772 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.772 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.772 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.772 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.772 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.772 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.772 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.772 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.772 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.772 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.772 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.772 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.773 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.773 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.773 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.773 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.773 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.773 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.773 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.773 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.773 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.773 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.773 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.773 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.773 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.773 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.773 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.774 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.774 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.774 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.774 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.774 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.774 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.774 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.774 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.774 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.774 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.774 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.774 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.774 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.774 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.774 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.774 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.775 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.775 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.775 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.775 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.775 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82026/exe 2024-02-05T07:52:38.775 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.775 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.775 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.775 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.775 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.775 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.775 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.775 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.775 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.775 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.776 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.776 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_misc_pp 2024-02-05T07:52:38.776 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.776 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.776 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82116 2024-02-05T07:52:38.776 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_misc_pp on pid 82116' 2024-02-05T07:52:38.776 INFO:tasks.workunit.client.0.smithi131.stdout:test api_misc_pp on pid 82116 2024-02-05T07:52:38.782 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82116 2024-02-05T07:52:38.782 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.782 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.783 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-02-05T07:52:38.783 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.783 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.783 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.783 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.783 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.783 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.783 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.783 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.783 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.783 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.783 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.783 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.783 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.783 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.784 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.784 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.784 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.784 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.784 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.784 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.784 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.784 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.784 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.784 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.784 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.784 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.784 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.784 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.784 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.785 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82048/exe 2024-02-05T07:52:38.785 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.785 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.785 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.785 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.785 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.785 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.785 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.785 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.785 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.785 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.785 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.785 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.785 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.785 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.785 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.786 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.786 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.786 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.786 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.786 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.786 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.786 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.786 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.786 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.786 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.786 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.786 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.786 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.786 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.786 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_list.log 2024-02-05T07:52:38.787 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.787 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.787 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.787 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.787 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.787 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.787 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.787 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.787 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.787 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.787 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.787 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.787 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.787 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.787 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.788 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.788 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.788 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.788 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.788 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.788 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.788 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.788 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.788 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.789 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.789 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.789 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.789 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.789 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.789 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.789 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_list.xml 2024-02-05T07:52:38.789 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_list: /' 2024-02-05T07:52:38.789 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock.xml 2024-02-05T07:52:38.789 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.789 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.789 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.789 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.789 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_lock: /' 2024-02-05T07:52:38.790 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.791 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.791 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.791 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.791 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.791 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.791 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.791 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.791 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.791 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_tier_pp 2024-02-05T07:52:38.791 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_tier_pp' 2024-02-05T07:52:38.791 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_lock.log 2024-02-05T07:52:38.791 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.791 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.791 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.791 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.791 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.792 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.792 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.792 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.792 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.792 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.792 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.792 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.792 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.792 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.792 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.792 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.792 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.792 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.792 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_tier_pp 2024-02-05T07:52:38.793 INFO:tasks.workunit.client.0.smithi131.stdout:test api_tier_pp on pid 82140 2024-02-05T07:52:38.797 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_tier_pp 2024-02-05T07:52:38.797 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82140 2024-02-05T07:52:38.797 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_tier_pp on pid 82140' 2024-02-05T07:52:38.797 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82140 2024-02-05T07:52:38.797 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.797 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.797 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-02-05T07:52:38.797 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.797 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.797 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.798 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.798 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.798 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.798 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_pool 2024-02-05T07:52:38.798 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_pool' 2024-02-05T07:52:38.798 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.798 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.798 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.798 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.798 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.798 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.798 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.798 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.798 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.798 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.798 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.799 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.799 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.799 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.799 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.799 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.799 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.799 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.799 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.799 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.799 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.799 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.799 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.799 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.800 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_pool 2024-02-05T07:52:38.800 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.800 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.800 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.800 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.800 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.800 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.800 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.800 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.800 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.800 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.800 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.800 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.800 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.800 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.801 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.801 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.801 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.801 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.801 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.801 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.801 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.801 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.801 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.801 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.801 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.801 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.801 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.801 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.801 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.802 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.802 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.802 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.802 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.802 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.802 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.802 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.802 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.802 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.802 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.802 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.802 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.802 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.802 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.802 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.803 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.803 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.803 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.803 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.803 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.803 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.803 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.803 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.803 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.803 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.803 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.803 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.803 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_pool 2024-02-05T07:52:38.804 INFO:tasks.workunit.client.0.smithi131.stdout:test api_pool on pid 82183 2024-02-05T07:52:38.807 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82183 2024-02-05T07:52:38.807 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_pool on pid 82183' 2024-02-05T07:52:38.808 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82183 2024-02-05T07:52:38.808 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.808 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.808 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-02-05T07:52:38.808 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.808 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.808 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.808 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.808 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.808 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.808 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.808 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.808 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.808 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.809 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.809 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.809 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.809 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.809 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.809 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.809 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.809 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.809 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.809 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.809 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.809 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.809 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.809 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.809 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.810 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.810 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.810 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.810 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.810 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.810 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.810 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.810 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.810 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.810 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.810 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.810 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.810 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.810 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.810 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.811 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.811 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.811 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.811 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.811 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82066/exe 2024-02-05T07:52:38.811 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.811 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_snapshots 2024-02-05T07:52:38.811 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.811 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_snapshots' 2024-02-05T07:52:38.811 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.811 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.811 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.811 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.811 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.811 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.811 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.812 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.812 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.812 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.812 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.812 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.812 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.812 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.812 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.812 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.812 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.812 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.812 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.812 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.812 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.813 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.813 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.813 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.813 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.813 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.813 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.813 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.813 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.813 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.813 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.813 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.813 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.813 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.813 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.813 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.814 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.814 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.814 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.814 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_snapshots 2024-02-05T07:52:38.814 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.814 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.814 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.814 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.814 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.814 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.814 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.814 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.814 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.814 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.814 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.814 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82090/exe 2024-02-05T07:52:38.815 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-02-05T07:52:38.815 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_lock_pp: /' 2024-02-05T07:52:38.815 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.815 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.815 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.815 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.815 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.815 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.815 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.815 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.815 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.815 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.815 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.815 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.815 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.816 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.816 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.816 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.816 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.816 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.816 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.816 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.816 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.816 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.816 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.816 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.816 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.816 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.816 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.816 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.816 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.817 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.817 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.817 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_snapshots 2024-02-05T07:52:38.817 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82232 2024-02-05T07:52:38.817 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_snapshots on pid 82232' 2024-02-05T07:52:38.817 INFO:tasks.workunit.client.0.smithi131.stdout:test api_snapshots on pid 82232 2024-02-05T07:52:38.821 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82232 2024-02-05T07:52:38.821 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.821 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.821 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.821 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.821 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.821 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.821 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.822 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.822 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_snapshots_pp 2024-02-05T07:52:38.822 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.822 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.822 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.822 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock_pp.xml 2024-02-05T07:52:38.822 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.822 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.822 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.822 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.822 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.822 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.822 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_snapshots_pp' 2024-02-05T07:52:38.822 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc.xml 2024-02-05T07:52:38.823 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-02-05T07:52:38.823 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.823 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.823 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.823 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.823 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.823 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.823 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.823 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.823 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.823 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.823 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.823 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.823 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.823 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.823 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.824 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.824 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.824 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.824 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.824 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.824 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.824 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.824 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.824 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.824 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.824 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.824 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.824 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.824 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.825 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.825 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.825 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.825 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.825 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_misc: /' 2024-02-05T07:52:38.825 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.825 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_snapshots_pp 2024-02-05T07:52:38.825 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.825 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.825 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.825 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.825 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.825 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.825 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.825 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.826 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.826 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.826 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.826 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.826 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.826 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.826 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.826 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.826 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.826 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.826 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.826 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.826 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.826 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.826 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.827 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.827 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.827 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_misc.log 2024-02-05T07:52:38.827 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.827 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.827 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.827 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.827 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.827 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.827 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.827 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.827 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.827 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.827 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.827 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.827 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.828 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.828 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.828 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.828 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.828 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.828 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.828 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.828 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.828 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.828 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.828 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.828 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.828 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.828 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.828 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.829 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.829 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.829 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.829 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.829 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.829 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.829 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.829 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.829 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.829 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.829 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.829 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.829 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.830 INFO:tasks.workunit.client.0.smithi131.stdout:test api_snapshots_pp on pid 82262 2024-02-05T07:52:38.834 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_snapshots_pp 2024-02-05T07:52:38.834 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82262 2024-02-05T07:52:38.834 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_snapshots_pp on pid 82262' 2024-02-05T07:52:38.834 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82262 2024-02-05T07:52:38.834 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.834 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.834 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-02-05T07:52:38.834 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.834 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_stat 2024-02-05T07:52:38.834 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_stat' 2024-02-05T07:52:38.834 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_stat 2024-02-05T07:52:38.834 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.834 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.835 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.835 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.835 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.835 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.835 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.835 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.835 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.835 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.835 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.835 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.835 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.835 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.835 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.835 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.836 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.836 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.836 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.836 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.836 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.836 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.836 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.836 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.836 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.836 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.836 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.836 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.836 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.836 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.836 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.837 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.837 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.837 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.837 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.837 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.837 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.837 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.837 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.837 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.837 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.837 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.837 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.837 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.837 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.837 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.838 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.838 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.838 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.838 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.838 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.838 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.838 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.838 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.838 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.838 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.838 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.838 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.838 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.838 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.838 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.839 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.839 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.839 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.839 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.839 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.839 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.839 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.839 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.839 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.839 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.839 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.839 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.839 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.839 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.839 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.840 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.840 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.840 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.840 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82116/exe 2024-02-05T07:52:38.840 INFO:tasks.workunit.client.0.smithi131.stdout:test api_stat on pid 82277 2024-02-05T07:52:38.844 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_stat 2024-02-05T07:52:38.844 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82277 2024-02-05T07:52:38.844 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_stat on pid 82277' 2024-02-05T07:52:38.844 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82277 2024-02-05T07:52:38.844 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.844 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.844 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.844 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.844 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82140/exe 2024-02-05T07:52:38.845 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.845 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.845 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.845 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.845 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.845 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.845 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.845 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.845 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-02-05T07:52:38.845 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.845 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.845 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.845 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.845 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.846 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.846 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.846 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.846 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.846 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.846 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.846 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.846 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.846 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.846 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.846 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.846 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.846 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.846 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.846 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.846 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.847 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.847 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc_pp.xml 2024-02-05T07:52:38.847 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.847 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.847 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.847 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.847 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.847 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.847 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.847 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.847 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.847 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.847 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.847 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.847 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.847 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.848 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.848 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.848 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.848 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.848 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.848 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.848 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.848 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.848 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.848 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.848 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.848 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-02-05T07:52:38.848 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.848 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.848 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.849 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.849 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.849 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.849 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.849 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.849 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.849 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.849 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.849 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_stat_pp 2024-02-05T07:52:38.849 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.849 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.849 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_stat_pp' 2024-02-05T07:52:38.849 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.849 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.849 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.849 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.850 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.850 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.850 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.850 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.850 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.850 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_tier_pp: /' 2024-02-05T07:52:38.850 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.850 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.850 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_tier_pp.xml 2024-02-05T07:52:38.850 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.850 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.850 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.850 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.850 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.850 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-02-05T07:52:38.850 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_misc_pp: /' 2024-02-05T07:52:38.851 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82183/exe 2024-02-05T07:52:38.851 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.851 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.851 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.851 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.851 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.851 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.851 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.851 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.851 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.851 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.851 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.851 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.851 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.851 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.851 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.852 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.852 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.852 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.852 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.852 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.852 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.852 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.852 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.852 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.852 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.852 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.852 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.852 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.852 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.852 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.853 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.853 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_pool.log 2024-02-05T07:52:38.853 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_pool: /' 2024-02-05T07:52:38.853 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_pool.xml 2024-02-05T07:52:38.853 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.853 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.853 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.853 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.853 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.853 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.853 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.853 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.853 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.853 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.853 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.853 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.854 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.855 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.855 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.855 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.855 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.855 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.855 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.855 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.855 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.855 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.855 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.855 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.855 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.855 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.855 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.855 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.856 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.856 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.856 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.856 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.856 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.856 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.856 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.856 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.856 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.856 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.856 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.856 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.856 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.856 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.856 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.856 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.857 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.857 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.857 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.857 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.857 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.857 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.857 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.857 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.857 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.857 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.857 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.857 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.857 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.857 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.857 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.857 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.858 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.858 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.858 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.858 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.858 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_stat_pp 2024-02-05T07:52:38.858 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.858 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.858 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.858 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.858 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.858 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.858 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.858 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.858 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.858 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.859 INFO:tasks.workunit.client.0.smithi131.stdout:test api_stat_pp on pid 82346 2024-02-05T07:52:38.860 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_stat_pp 2024-02-05T07:52:38.860 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82346 2024-02-05T07:52:38.860 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_stat_pp on pid 82346' 2024-02-05T07:52:38.860 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82346 2024-02-05T07:52:38.860 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.860 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.860 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_watch_notify 2024-02-05T07:52:38.860 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_watch_notify' 2024-02-05T07:52:38.860 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.860 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-02-05T07:52:38.860 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.861 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_watch_notify 2024-02-05T07:52:38.861 INFO:tasks.workunit.client.0.smithi131.stdout:test api_watch_notify on pid 82357 2024-02-05T07:52:38.863 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_watch_notify 2024-02-05T07:52:38.863 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82357 2024-02-05T07:52:38.863 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_watch_notify on pid 82357' 2024-02-05T07:52:38.863 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82357 2024-02-05T07:52:38.863 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.863 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.863 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-02-05T07:52:38.863 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_watch_notify_pp 2024-02-05T07:52:38.863 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_watch_notify_pp' 2024-02-05T07:52:38.863 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.863 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.863 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.863 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.863 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.864 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.864 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.864 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.864 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_watch_notify_pp 2024-02-05T07:52:38.864 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.864 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.864 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.864 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.864 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.864 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.864 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.864 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.864 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.864 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.864 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.865 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.865 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.865 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.865 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.865 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.865 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.865 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.865 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.865 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.865 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.865 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.865 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.865 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.866 INFO:tasks.workunit.client.0.smithi131.stdout:test api_watch_notify_pp on pid 82372 2024-02-05T07:52:38.870 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_watch_notify_pp 2024-02-05T07:52:38.870 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.870 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.870 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.870 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82372 2024-02-05T07:52:38.870 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.870 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_watch_notify_pp on pid 82372' 2024-02-05T07:52:38.870 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.870 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82372 2024-02-05T07:52:38.870 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.870 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.870 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.870 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.870 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.871 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.871 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.871 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.871 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.871 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.871 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.871 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.871 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.871 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82232/exe 2024-02-05T07:52:38.871 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.871 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.871 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.871 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.871 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.871 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.872 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.872 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.872 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.872 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.872 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.872 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.872 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.872 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.872 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.872 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.872 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.872 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.872 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.872 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.872 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.873 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.873 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.873 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-02-05T07:52:38.873 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.873 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.873 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.873 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.873 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.873 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.873 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.873 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.873 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.873 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.873 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.873 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.874 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_cmd 2024-02-05T07:52:38.874 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.874 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.874 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.874 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.874 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots.xml 2024-02-05T07:52:38.874 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_cmd' 2024-02-05T07:52:38.874 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_snapshots: /' 2024-02-05T07:52:38.874 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.874 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.874 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.874 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.874 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.874 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.874 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.874 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.875 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.875 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.875 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.875 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.875 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.875 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.875 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.875 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.875 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.875 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.875 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.875 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.875 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_cmd 2024-02-05T07:52:38.875 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.875 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82277/exe 2024-02-05T07:52:38.875 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.876 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-02-05T07:52:38.876 INFO:tasks.workunit.client.0.smithi131.stdout:test api_cmd on pid 82397 2024-02-05T07:52:38.883 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_cmd 2024-02-05T07:52:38.884 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82397 2024-02-05T07:52:38.884 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_cmd on pid 82397' 2024-02-05T07:52:38.884 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82397 2024-02-05T07:52:38.884 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.884 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.884 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.884 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.884 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.884 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.884 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.884 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.884 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_cmd_pp 2024-02-05T07:52:38.884 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.884 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.885 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.885 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.885 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_cmd_pp' 2024-02-05T07:52:38.885 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-02-05T07:52:38.885 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.885 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.885 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.885 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.885 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.885 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.885 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.885 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.885 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.885 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.885 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.885 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.886 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.886 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.886 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.886 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.886 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.886 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.886 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.886 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.886 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.886 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.886 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.886 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_stat: /' 2024-02-05T07:52:38.886 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.886 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.886 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat.xml 2024-02-05T07:52:38.887 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.887 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.887 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.887 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.887 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.887 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.887 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.887 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.887 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.887 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.887 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.887 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.887 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.887 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.887 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.887 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.888 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_stat.log 2024-02-05T07:52:38.888 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.888 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.888 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.888 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.888 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.888 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.888 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.888 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.888 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.889 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.889 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.889 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.889 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.889 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.889 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.889 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.889 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.889 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.889 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.889 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.889 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.889 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.889 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.889 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.889 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.890 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.890 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.890 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.890 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.890 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.890 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.890 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.890 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.890 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.890 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.890 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.890 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.890 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.890 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.890 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.891 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.891 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.891 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.891 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.891 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.891 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.891 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.891 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.891 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.891 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.891 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.891 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.891 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.891 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.891 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.891 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.892 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.892 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_cmd_pp 2024-02-05T07:52:38.892 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.892 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.892 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.892 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82262/exe 2024-02-05T07:52:38.892 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.892 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.892 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.892 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.892 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.892 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.892 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.892 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.892 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.893 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.893 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.893 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.893 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.893 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.893 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.893 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.893 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.893 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.893 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.893 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.893 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.893 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.893 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.893 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.894 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.894 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.894 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.894 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.894 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.894 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.894 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.894 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.894 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.894 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.894 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.894 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.894 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.894 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.894 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.895 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.895 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.895 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.895 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.895 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.895 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.895 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.895 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.895 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.895 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.895 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.895 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.895 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.895 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.895 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.895 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-02-05T07:52:38.896 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots_pp.xml 2024-02-05T07:52:38.896 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-02-05T07:52:38.896 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.896 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.896 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.896 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.896 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.896 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.896 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.896 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.896 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.896 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.896 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.896 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.896 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.897 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.897 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.897 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.897 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.897 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.897 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.897 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.897 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.897 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82346/exe 2024-02-05T07:52:38.897 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.897 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.897 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.897 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.897 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.897 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.897 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.898 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.898 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.898 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.898 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.898 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.898 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.898 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.898 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.898 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.898 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.898 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.898 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.898 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.898 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.898 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.898 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.899 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.899 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.899 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.899 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.899 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.899 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.899 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.899 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.899 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.899 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat_pp.xml 2024-02-05T07:52:38.899 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-02-05T07:52:38.899 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.899 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.899 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.899 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.900 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.900 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.900 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.900 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.900 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.900 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.900 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.900 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.900 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.900 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.900 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.900 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.900 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.900 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.901 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.901 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.901 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.901 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.901 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.901 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_stat_pp: /' 2024-02-05T07:52:38.901 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.901 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.901 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.901 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.901 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.901 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.901 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_cmd_pp 2024-02-05T07:52:38.902 INFO:tasks.workunit.client.0.smithi131.stdout:test api_cmd_pp on pid 82468 2024-02-05T07:52:38.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82468 2024-02-05T07:52:38.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_cmd_pp on pid 82468' 2024-02-05T07:52:38.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82468 2024-02-05T07:52:38.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.908 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.908 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.908 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.909 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.909 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.909 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.909 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.909 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.909 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.909 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-02-05T07:52:38.909 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_service 2024-02-05T07:52:38.909 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_service' 2024-02-05T07:52:38.909 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.909 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.909 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.909 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.909 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.909 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.909 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_service 2024-02-05T07:52:38.910 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.910 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.910 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.910 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.910 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.910 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.910 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.910 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.910 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.910 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.910 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.910 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.910 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.910 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.910 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.911 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.911 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.911 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.911 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.911 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.911 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.911 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.911 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.911 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.911 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.911 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.911 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.911 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.911 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.911 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.911 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.912 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.912 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.912 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.912 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.912 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.912 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.912 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.912 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.912 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.912 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.912 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.912 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.912 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.912 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.912 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.912 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.913 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.913 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.913 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.913 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.913 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.913 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.913 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.913 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.913 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.913 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.913 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.913 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.913 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.913 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.913 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.913 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.914 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.914 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.914 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.914 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.914 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.914 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.914 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.914 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.914 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.914 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.914 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.914 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.914 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.914 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.914 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.914 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.915 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.915 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.915 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.915 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.915 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.915 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.915 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.915 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.915 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.915 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.915 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.915 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.916 INFO:tasks.workunit.client.0.smithi131.stdout:test api_service on pid 82500 2024-02-05T07:52:38.922 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.922 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.922 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.922 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.923 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.923 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.923 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.923 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.923 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.923 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.923 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.923 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.923 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_service 2024-02-05T07:52:38.923 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.923 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.923 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.923 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.923 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82500 2024-02-05T07:52:38.923 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_service on pid 82500' 2024-02-05T07:52:38.924 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82500 2024-02-05T07:52:38.924 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.924 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.924 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.924 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-02-05T07:52:38.924 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.924 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.924 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.925 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.925 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.925 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.925 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.925 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.925 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.925 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.925 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.925 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.925 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.925 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.925 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.925 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.925 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.925 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.925 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.926 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.926 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.926 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.926 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.926 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.926 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.926 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.926 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.926 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.926 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.926 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.926 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.926 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.926 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.926 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82357/exe 2024-02-05T07:52:38.926 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.927 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.927 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.927 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.927 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.927 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.927 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.927 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.927 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.927 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.927 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_service_pp 2024-02-05T07:52:38.927 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_service_pp' 2024-02-05T07:52:38.927 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.927 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.927 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.927 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.927 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.928 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.928 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.928 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.928 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.928 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.928 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.928 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.928 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.928 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.928 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.928 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.928 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.928 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.928 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.928 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.928 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.929 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.929 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.929 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.929 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.929 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.929 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.929 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.929 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.929 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify.xml 2024-02-05T07:52:38.929 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-02-05T07:52:38.929 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.929 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.929 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.929 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.929 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.929 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.930 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.930 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.930 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.930 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.930 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.930 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.930 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.930 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.930 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.930 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.930 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.930 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.930 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.930 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.930 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.931 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.931 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_watch_notify: /' 2024-02-05T07:52:38.931 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.931 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_service_pp 2024-02-05T07:52:38.931 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.931 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82372/exe 2024-02-05T07:52:38.931 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.931 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.931 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.931 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.931 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.931 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.931 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.931 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.931 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.931 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.932 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.932 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.932 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.932 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.932 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.932 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.932 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.932 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.932 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.932 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.932 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.932 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.933 INFO:tasks.workunit.client.0.smithi131.stdout:test api_service_pp on pid 82530 2024-02-05T07:52:38.935 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.936 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.936 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.936 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.936 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.936 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.936 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.936 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.936 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.936 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.936 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.936 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_service_pp 2024-02-05T07:52:38.936 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82530 2024-02-05T07:52:38.936 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_service_pp on pid 82530' 2024-02-05T07:52:38.936 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82530 2024-02-05T07:52:38.937 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.937 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.937 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-02-05T07:52:38.937 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.937 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.937 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.937 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.937 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.937 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.937 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.937 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.937 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.937 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.937 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.937 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.937 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.938 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.938 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.938 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.938 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.938 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.938 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.938 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.938 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.938 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.938 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.938 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.938 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.938 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.938 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.938 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.938 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.939 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.939 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.939 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_c_write_operations 2024-02-05T07:52:38.939 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.939 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.939 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.939 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.939 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.939 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.939 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.939 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.939 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.939 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.939 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.939 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify_pp.xml 2024-02-05T07:52:38.939 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82397/exe 2024-02-05T07:52:38.940 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-02-05T07:52:38.940 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_c_write_operations' 2024-02-05T07:52:38.940 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.940 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.940 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.940 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.940 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.940 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.940 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.940 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.940 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.940 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.940 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.940 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.940 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.940 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.941 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.941 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.941 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.941 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-02-05T07:52:38.941 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.941 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.941 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.941 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.941 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.941 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.941 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.941 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.941 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.941 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.941 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.941 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.942 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.942 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.942 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.942 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.942 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.942 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.942 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_c_write_operations 2024-02-05T07:52:38.942 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.942 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.942 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.942 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.942 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.942 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.942 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.942 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_cmd.log 2024-02-05T07:52:38.942 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_cmd: /' 2024-02-05T07:52:38.943 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.943 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.943 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.943 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.943 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.943 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.943 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.943 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.943 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.943 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd.xml 2024-02-05T07:52:38.943 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_c_write_operations 2024-02-05T07:52:38.943 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82568 2024-02-05T07:52:38.944 INFO:tasks.workunit.client.0.smithi131.stdout:test api_c_write_operations on pid 82568 2024-02-05T07:52:38.947 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_c_write_operations on pid 82568' 2024-02-05T07:52:38.948 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82568 2024-02-05T07:52:38.948 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.948 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.948 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.948 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.948 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.948 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.948 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.948 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.948 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.948 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.948 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.948 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.948 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.949 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.949 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.949 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.949 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.949 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.949 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.949 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.949 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.949 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.949 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-02-05T07:52:38.949 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.949 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.949 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.949 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.949 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_c_read_operations 2024-02-05T07:52:38.950 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.950 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.950 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.950 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_c_read_operations' 2024-02-05T07:52:38.950 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.950 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.950 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.950 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.950 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.950 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.950 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.950 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.950 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.950 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.950 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.950 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.951 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.951 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.951 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.951 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.951 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.951 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.951 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.951 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.951 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.951 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.951 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.951 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.951 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.951 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.951 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.952 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.952 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.952 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.952 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.952 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.952 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.952 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.952 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.952 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.952 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.952 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.952 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.952 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.952 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.952 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.952 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.953 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.953 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.953 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.953 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.953 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.953 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.953 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82468/exe 2024-02-05T07:52:38.953 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.953 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.953 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.953 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.953 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.953 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.953 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.953 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.954 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.954 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.954 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.954 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.954 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.954 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.954 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.954 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.954 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.954 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.954 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.954 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.954 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.954 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.954 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.955 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.955 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.955 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.955 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.955 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.955 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.955 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-02-05T07:52:38.955 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-02-05T07:52:38.955 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.955 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.955 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.955 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.955 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.955 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.955 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd_pp.xml 2024-02-05T07:52:38.955 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.956 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.956 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.956 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.956 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.956 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.956 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.956 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.956 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.956 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.956 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.956 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.956 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.956 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.956 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.956 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.957 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.957 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.957 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.957 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.957 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.957 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.957 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.957 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.957 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.957 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.957 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.957 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.957 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.957 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.957 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.957 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.958 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.958 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.958 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.958 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.958 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.958 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.958 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.958 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.958 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.958 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.958 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.958 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.958 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.958 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.958 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.958 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.959 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.959 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.959 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.959 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.959 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.959 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_c_read_operations 2024-02-05T07:52:38.959 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.959 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.959 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.959 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.959 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.959 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.959 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.959 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.959 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.959 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.960 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.960 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.960 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82530/exe 2024-02-05T07:52:38.960 INFO:tasks.workunit.client.0.smithi131.stdout:test api_c_read_operations on pid 82618 2024-02-05T07:52:38.961 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_c_read_operations 2024-02-05T07:52:38.961 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82618 2024-02-05T07:52:38.961 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_c_read_operations on pid 82618' 2024-02-05T07:52:38.961 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82618 2024-02-05T07:52:38.961 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.961 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.961 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_cls_remote_reads 2024-02-05T07:52:38.961 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_cls_remote_reads' 2024-02-05T07:52:38.961 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.961 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-02-05T07:52:38.962 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_cls_remote_reads 2024-02-05T07:52:38.962 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.962 INFO:tasks.workunit.client.0.smithi131.stdout:test api_cls_remote_reads on pid 82635 2024-02-05T07:52:38.964 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.964 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.964 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.964 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.964 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.964 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.964 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.964 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.964 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.965 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.965 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service_pp.xml 2024-02-05T07:52:38.965 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_cls_remote_reads 2024-02-05T07:52:38.965 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82635 2024-02-05T07:52:38.965 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_cls_remote_reads on pid 82635' 2024-02-05T07:52:38.965 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82635 2024-02-05T07:52:38.965 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.965 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.965 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.965 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.965 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.965 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s list_parallel 2024-02-05T07:52:38.965 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' list_parallel' 2024-02-05T07:52:38.965 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_service_pp: /' 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cls_remote_reads.xml 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.966 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.967 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.967 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.967 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.967 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.967 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.967 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.967 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.967 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.967 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.967 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.967 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.967 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.967 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.967 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.968 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo list_parallel 2024-02-05T07:52:38.968 INFO:tasks.workunit.client.0.smithi131.stdout:test list_parallel on pid 82645 2024-02-05T07:52:38.974 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.974 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.975 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.975 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=list_parallel 2024-02-05T07:52:38.975 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.975 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.975 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.975 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.975 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.975 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.975 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82645 2024-02-05T07:52:38.975 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test list_parallel on pid 82645' 2024-02-05T07:52:38.975 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82645 2024-02-05T07:52:38.975 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.975 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.975 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s open_pools_parallel 2024-02-05T07:52:38.975 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.976 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' open_pools_parallel' 2024-02-05T07:52:38.976 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.976 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.976 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.976 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.976 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.976 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.976 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.976 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.976 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.976 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.976 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.976 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.976 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.977 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.977 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.977 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.977 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.977 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.977 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.977 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.977 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.977 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.977 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.977 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.977 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.977 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.977 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.977 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.978 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.978 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.978 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.978 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.978 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.978 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.978 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.978 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.978 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.978 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.978 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.978 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.978 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.978 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.978 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.979 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.979 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.979 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.979 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.979 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.979 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.979 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.979 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.979 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.979 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.979 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.979 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.979 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.979 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.979 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.979 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.980 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.980 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82500/exe 2024-02-05T07:52:38.980 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.980 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.980 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.980 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.980 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.980 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.980 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.980 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.980 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.980 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.980 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.980 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.980 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.981 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.981 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.981 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.981 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.981 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.981 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.981 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.981 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.981 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.981 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.981 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.981 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.981 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.981 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.981 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.981 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.982 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.982 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.982 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.982 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.982 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.982 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.982 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.982 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.982 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.982 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.982 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.982 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.982 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.982 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.982 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.983 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.983 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.983 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.983 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.983 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.983 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.983 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.983 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.983 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.983 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.983 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.983 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.983 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.983 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.983 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service.xml 2024-02-05T07:52:38.983 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.984 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.984 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.984 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.984 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.984 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-02-05T07:52:38.984 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_service: /' 2024-02-05T07:52:38.984 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_service.log 2024-02-05T07:52:38.984 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.984 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.984 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.984 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.984 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.984 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.984 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.985 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo open_pools_parallel 2024-02-05T07:52:38.985 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.985 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.985 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.985 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.985 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.985 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.985 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.985 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.985 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.985 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.985 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.985 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.986 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.986 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.986 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.986 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.986 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.986 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.986 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.986 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:38.986 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:38.986 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:38.986 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.986 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:38.986 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.986 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:38.986 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:38.987 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.987 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.987 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.987 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.987 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.987 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.987 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.987 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:38.987 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:38.987 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.987 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:38.987 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:38.987 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.987 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:38.988 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.988 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:38.988 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:38.988 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.988 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:38.988 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.988 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:38.988 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:38.988 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:38.988 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.988 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.988 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.988 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.988 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.988 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.989 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.989 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.989 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.989 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.989 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.989 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.989 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.989 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.989 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.989 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.989 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82568/exe 2024-02-05T07:52:38.990 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.990 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.990 INFO:tasks.workunit.client.0.smithi131.stdout:test open_pools_parallel on pid 82698 2024-02-05T07:52:38.993 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=open_pools_parallel 2024-02-05T07:52:38.993 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82698 2024-02-05T07:52:38.993 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test open_pools_parallel on pid 82698' 2024-02-05T07:52:38.993 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82698 2024-02-05T07:52:38.993 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:38.993 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:52:38.993 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:38.993 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s delete_pools_parallel 2024-02-05T07:52:38.993 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' delete_pools_parallel' 2024-02-05T07:52:38.994 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:38.994 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:38.994 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:38.994 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:38.994 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:38.994 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:38.994 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:38.994 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-02-05T07:52:38.994 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:38.994 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:38.994 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:38.994 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_write_operations.xml 2024-02-05T07:52:38.994 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:38.994 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:38.994 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:38.995 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.995 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:38.995 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.995 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:38.995 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:38.995 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:38.995 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.995 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:38.995 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.995 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:38.995 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:38.995 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:38.995 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.995 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:38.995 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.995 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:38.996 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.996 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:38.996 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:38.996 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:38.996 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.996 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:38.996 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.996 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:38.996 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:38.996 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.996 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:38.996 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.996 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:38.996 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:38.996 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:38.996 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.997 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:38.997 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.997 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:38.997 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:38.997 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:38.997 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:38.997 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.997 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:38.997 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-02-05T07:52:38.997 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-02-05T07:52:38.997 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:38.997 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:38.997 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.997 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.997 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.997 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.998 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:38.998 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.998 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:38.998 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.998 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:38.998 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.998 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:38.998 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:38.998 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.998 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:38.998 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.998 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:38.998 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.998 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:38.998 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:38.998 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:38.999 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:38.999 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:38.999 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:38.999 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:38.999 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo delete_pools_parallel 2024-02-05T07:52:38.999 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:38.999 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:38.999 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.999 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:38.999 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:38.999 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:38.999 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:38.999 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:38.999 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:38.999 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:38.999 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.000 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.000 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.000 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.000 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.000 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.000 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.000 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.000 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.000 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.000 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.000 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.000 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.000 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.000 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.000 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.001 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.001 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:39.001 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:39.001 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:39.001 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:39.001 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.001 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:39.001 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.001 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:39.001 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:39.001 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.001 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:39.001 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:39.001 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:39.001 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.001 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:39.002 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.002 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:39.002 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:39.002 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:39.002 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.002 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:39.002 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.002 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:39.002 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:39.002 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:39.002 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.002 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:39.002 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.002 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:39.002 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.003 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82618/exe 2024-02-05T07:52:39.003 INFO:tasks.workunit.client.0.smithi131.stdout:test delete_pools_parallel on pid 82745 2024-02-05T07:52:39.006 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=delete_pools_parallel 2024-02-05T07:52:39.006 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82745 2024-02-05T07:52:39.006 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test delete_pools_parallel on pid 82745' 2024-02-05T07:52:39.007 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82745 2024-02-05T07:52:39.007 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:52:39.007 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:39.007 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.007 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.007 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.007 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.007 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.007 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.007 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.007 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.007 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.007 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.007 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.007 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.008 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.008 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.008 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.008 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-02-05T07:52:39.008 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.008 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.008 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.008 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.008 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.008 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.008 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.008 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:39.008 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:39.008 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:39.008 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.009 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:39.009 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.009 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:39.009 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:39.009 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.009 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:39.009 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.009 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:39.009 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.009 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:39.009 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.009 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:39.009 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:39.009 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.009 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.010 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.010 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.010 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.010 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.010 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.010 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.010 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:39.010 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.010 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:39.010 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.010 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s cls 2024-02-05T07:52:39.010 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' cls' 2024-02-05T07:52:39.010 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:39.010 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:39.010 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:39.011 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:39.011 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:39.011 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:39.011 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.011 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:39.011 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.011 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:39.011 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:39.011 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.011 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.011 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo cls 2024-02-05T07:52:39.011 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:39.011 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-02-05T07:52:39.011 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:39.011 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.011 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:39.012 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.012 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:39.012 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:39.012 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82635/exe 2024-02-05T07:52:39.012 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.012 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.012 INFO:tasks.workunit.client.0.smithi131.stdout:test cls on pid 82768 2024-02-05T07:52:39.015 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=cls 2024-02-05T07:52:39.015 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82768 2024-02-05T07:52:39.015 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test cls on pid 82768' 2024-02-05T07:52:39.015 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.015 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82768 2024-02-05T07:52:39.015 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:52:39.016 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.016 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:39.016 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.016 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.016 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.016 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.016 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.016 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.016 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.016 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.016 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.016 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.016 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.016 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.017 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.017 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.017 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.017 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.017 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.017 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.017 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.017 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.017 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.017 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.017 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.017 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s cmd 2024-02-05T07:52:39.017 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.017 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.017 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.017 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.018 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.018 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.018 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' cmd' 2024-02-05T07:52:39.018 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-02-05T07:52:39.018 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-02-05T07:52:39.018 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_read_operations.xml 2024-02-05T07:52:39.018 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-02-05T07:52:39.018 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:39.018 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cls_remote_reads.xml 2024-02-05T07:52:39.018 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:39.018 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:39.018 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:39.018 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.018 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:39.018 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:39.019 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:39.019 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:39.019 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:39.019 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.019 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:39.019 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.019 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:39.019 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:39.019 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.019 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:39.019 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.019 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:39.019 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.019 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.019 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.019 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:39.020 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.020 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:39.020 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.020 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82645/exe 2024-02-05T07:52:39.020 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:39.020 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:39.020 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:39.020 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.020 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:39.020 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.020 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:39.020 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:39.020 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.020 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:39.020 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.020 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:39.021 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.021 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:39.021 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.021 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:39.021 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:39.021 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.021 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:39.021 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-02-05T07:52:39.021 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.021 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.021 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.021 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.021 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.021 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.021 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.022 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.022 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.022 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.022 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.022 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:39.022 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.022 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:39.022 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:39.022 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo cmd 2024-02-05T07:52:39.022 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ list_parallel: /' 2024-02-05T07:52:39.022 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=cmd 2024-02-05T07:52:39.022 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:39.022 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82798 2024-02-05T07:52:39.023 INFO:tasks.workunit.client.0.smithi131.stdout:test cmd on pid 82798 2024-02-05T07:52:39.026 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test cmd on pid 82798' 2024-02-05T07:52:39.026 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82798 2024-02-05T07:52:39.026 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:52:39.026 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:39.026 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:39.026 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:39.026 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.026 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:39.026 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.027 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:39.027 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:39.027 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.027 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:39.027 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.027 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:39.027 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.027 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.027 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.027 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:39.027 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.027 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:39.027 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.027 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.027 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.027 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.028 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.028 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.028 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.028 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.028 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.028 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.028 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.028 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.028 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.028 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.028 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.028 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.028 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.028 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.029 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.029 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.029 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.029 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.029 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.029 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82698/exe 2024-02-05T07:52:39.029 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-list_parallel.xml 2024-02-05T07:52:39.029 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.029 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_list_parallel.log 2024-02-05T07:52:39.029 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.029 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.029 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.029 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.029 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.029 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.029 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-02-05T07:52:39.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-02-05T07:52:39.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-open_pools_parallel.xml 2024-02-05T07:52:39.030 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s handler_error 2024-02-05T07:52:39.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' handler_error' 2024-02-05T07:52:39.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-02-05T07:52:39.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:39.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:39.030 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:39.030 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:39.030 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:39.030 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:39.031 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:39.031 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:39.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:39.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:39.031 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:39.031 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:39.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:39.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:39.031 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:39.031 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:39.032 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.032 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:39.032 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.032 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:39.032 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.032 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:39.032 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo handler_error 2024-02-05T07:52:39.032 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:39.032 INFO:tasks.workunit.client.0.smithi131.stdout:test handler_error on pid 82837 2024-02-05T07:52:39.038 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=handler_error 2024-02-05T07:52:39.038 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82837 2024-02-05T07:52:39.039 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test handler_error on pid 82837' 2024-02-05T07:52:39.039 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82837 2024-02-05T07:52:39.039 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:52:39.039 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:39.039 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-02-05T07:52:39.039 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:39.039 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:39.039 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.039 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:39.039 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.039 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:39.039 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:39.039 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.039 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:39.040 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.040 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:39.040 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.040 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.040 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.040 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:39.040 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.040 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:39.040 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.040 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:39.040 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:39.040 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:39.040 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:39.040 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.040 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:39.041 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s io 2024-02-05T07:52:39.041 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' io' 2024-02-05T07:52:39.041 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:39.041 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:39.041 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:39.041 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.041 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:39.041 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.041 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:39.041 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:39.041 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.041 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:39.041 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.041 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:39.041 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.042 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.042 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.042 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:39.042 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.042 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:39.042 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.042 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:39.042 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:39.042 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:39.042 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.042 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:39.042 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.042 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:39.042 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:39.043 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.043 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:39.043 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:39.043 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:39.043 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.043 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:39.043 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.043 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:39.043 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:39.043 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:39.043 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.043 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:39.043 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.043 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:39.043 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:39.043 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:39.044 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.044 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:39.044 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.044 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:39.044 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.044 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:39.044 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:39.044 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:39.044 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.044 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:39.044 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.044 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:39.044 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:39.044 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.044 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:39.045 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.045 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:39.045 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.045 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:39.045 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.045 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:39.045 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:39.045 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo io 2024-02-05T07:52:39.045 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:39.045 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:39.045 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:39.045 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:39.045 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:39.045 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.045 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:39.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:39.046 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:39.046 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:39.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:39.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:39.046 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:39.046 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:39.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:39.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:39.046 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.046 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.047 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:39.047 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.047 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:39.047 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:39.047 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:39.047 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:39.047 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:39.047 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.047 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:39.047 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.047 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:39.047 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:39.047 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.047 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.048 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.048 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.048 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.048 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.048 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.048 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.048 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.048 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.048 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.048 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.048 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.048 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.048 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.048 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.048 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.049 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.049 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.049 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.049 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.049 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.049 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.049 INFO:tasks.workunit.client.0.smithi131.stdout:test io on pid 82876 2024-02-05T07:52:39.051 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:39.051 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:39.051 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:39.051 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.051 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:39.051 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.051 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:39.051 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:39.051 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.051 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:39.051 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.052 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:39.052 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=io 2024-02-05T07:52:39.052 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.052 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:39.052 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.052 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:39.052 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82876 2024-02-05T07:52:39.052 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test io on pid 82876' 2024-02-05T07:52:39.052 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82876 2024-02-05T07:52:39.052 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:39.052 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:52:39.052 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:39.052 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:39.052 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s ec_io 2024-02-05T07:52:39.052 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' ec_io' 2024-02-05T07:52:39.052 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82745/exe 2024-02-05T07:52:39.053 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.053 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:39.053 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:39.053 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:39.053 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.053 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:39.053 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.053 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:39.053 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:39.053 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.053 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.053 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:39.053 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.053 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:39.053 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:39.054 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.054 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.054 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.054 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.054 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.054 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.054 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.054 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.054 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.054 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.054 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-delete_pools_parallel.xml 2024-02-05T07:52:39.054 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:39.054 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:39.054 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:39.054 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.054 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:39.055 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.055 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:39.055 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:39.055 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.055 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:39.055 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.055 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:39.055 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.055 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:39.055 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.055 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:39.055 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:39.055 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:39.055 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:39.055 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:39.056 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.056 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:39.056 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.056 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:39.056 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:39.056 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:39.056 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.056 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:39.056 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.056 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:39.056 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:39.056 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:39.056 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.056 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:39.056 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.057 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:39.057 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.057 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:39.057 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.057 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo ec_io 2024-02-05T07:52:39.057 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-02-05T07:52:39.057 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-02-05T07:52:39.057 INFO:tasks.workunit.client.0.smithi131.stdout:test ec_io on pid 82911 2024-02-05T07:52:39.064 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=ec_io 2024-02-05T07:52:39.064 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82911 2024-02-05T07:52:39.064 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test ec_io on pid 82911' 2024-02-05T07:52:39.064 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82911 2024-02-05T07:52:39.065 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:52:39.065 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:39.065 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-02-05T07:52:39.065 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:39.065 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.065 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.065 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.065 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.065 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.065 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.065 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.065 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.065 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.065 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.065 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.065 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.066 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.066 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.066 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.066 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.066 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.066 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.066 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.066 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.066 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.066 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:39.066 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:39.066 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:39.066 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:39.066 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.066 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:39.066 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.067 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.067 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:39.067 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:39.067 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:39.067 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82837/exe 2024-02-05T07:52:39.067 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.067 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.067 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:39.067 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:39.067 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:39.067 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.067 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:39.067 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.067 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:39.067 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:39.068 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.068 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:39.068 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:39.068 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.068 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:39.068 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.068 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.068 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.068 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.068 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.068 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:39.068 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.068 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.068 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:39.068 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.069 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.069 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.069 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.069 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.069 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.069 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:39.069 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.069 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:39.069 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.069 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.069 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.069 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:39.069 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.069 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.069 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.070 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.070 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.070 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:39.070 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.070 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.070 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.070 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:39.070 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.070 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.070 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.070 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.070 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.070 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.070 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.070 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.070 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.071 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.071 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-02-05T07:52:39.071 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_neorados_handler_error 2024-02-05T07:52:39.071 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ handler_error: /' 2024-02-05T07:52:39.071 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:39.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s list 2024-02-05T07:52:39.071 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:39.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:39.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:39.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:39.072 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_neorados_handler_error.log 2024-02-05T07:52:39.072 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' list' 2024-02-05T07:52:39.072 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82768/exe 2024-02-05T07:52:39.072 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.072 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:39.072 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:39.072 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:39.072 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:39.072 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.072 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:39.072 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.072 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:39.072 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:39.072 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:39.072 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:39.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.073 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:39.073 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.073 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:39.073 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:39.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.073 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:39.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:39.073 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.074 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.074 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.074 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.074 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.074 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.074 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo list 2024-02-05T07:52:39.074 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_neorados_cls 2024-02-05T07:52:39.074 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ cls: /' 2024-02-05T07:52:39.074 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=list 2024-02-05T07:52:39.074 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82966 2024-02-05T07:52:39.074 INFO:tasks.workunit.client.0.smithi131.stdout:test list on pid 82966 2024-02-05T07:52:39.076 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test list on pid 82966' 2024-02-05T07:52:39.076 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82966 2024-02-05T07:52:39.076 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:52:39.076 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:39.076 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-02-05T07:52:39.076 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:39.077 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:39.077 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:39.077 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:39.077 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:39.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:39.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:39.077 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:39.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:39.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:39.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:39.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:39.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.078 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:39.078 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.078 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:39.078 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.078 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:39.078 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:39.078 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:39.078 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s ec_list 2024-02-05T07:52:39.078 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' ec_list' 2024-02-05T07:52:39.078 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:39.078 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.078 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:39.078 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.078 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:39.078 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:39.078 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:39.079 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:39.079 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo ec_list 2024-02-05T07:52:39.079 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.079 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:39.079 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.079 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:39.079 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:39.079 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.079 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:39.079 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.079 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:39.079 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.079 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.079 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.079 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:39.079 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.080 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:39.080 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.080 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:39.080 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:39.080 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:39.080 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.080 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:39.080 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.080 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:39.080 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:39.080 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.080 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:39.080 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.080 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:39.080 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.081 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.081 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.081 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:39.081 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.081 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:39.081 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.081 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:39.081 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:39.081 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:39.081 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:39.081 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.081 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:39.081 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.081 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:39.081 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:39.081 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.082 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_neorados_cls.log 2024-02-05T07:52:39.082 INFO:tasks.workunit.client.0.smithi131.stdout:test ec_list on pid 82999 2024-02-05T07:52:39.090 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=ec_list 2024-02-05T07:52:39.090 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82999 2024-02-05T07:52:39.090 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test ec_list on pid 82999' 2024-02-05T07:52:39.090 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=82999 2024-02-05T07:52:39.090 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:52:39.090 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:39.090 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.090 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.090 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.090 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.090 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.091 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.091 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.091 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.091 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.091 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.091 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.091 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.092 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.092 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.092 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:39.092 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:39.092 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:39.092 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.092 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:39.092 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.092 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:39.092 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:39.092 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:39.092 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.092 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:39.092 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.092 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:39.093 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:39.093 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:39.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:39.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:39.093 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.093 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-02-05T07:52:39.093 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:39.093 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:39.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:39.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:39.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:39.093 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:39.094 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:39.094 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:39.094 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.094 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:39.094 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.094 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:39.094 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:39.094 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.094 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:39.094 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.094 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:39.094 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.094 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.094 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.094 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:39.095 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.095 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:39.095 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.095 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s misc 2024-02-05T07:52:39.095 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' misc' 2024-02-05T07:52:39.095 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:39.095 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:39.095 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:39.095 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.095 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:39.095 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.095 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:39.095 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:39.095 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:39.095 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.096 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:39.096 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.096 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:39.096 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:39.096 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:39.096 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.096 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:39.096 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.096 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:39.096 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.096 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:39.096 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:39.096 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:39.096 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.096 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:39.096 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.097 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:39.097 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:39.097 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.097 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:39.097 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.097 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:39.097 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.097 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:39.097 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.097 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:39.097 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:39.097 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:39.097 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:39.097 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:39.097 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:39.097 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.098 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo misc 2024-02-05T07:52:39.099 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.099 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:39.099 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.099 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:39.099 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:39.099 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:39.099 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:39.099 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:39.099 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.099 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:39.099 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.099 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:39.099 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:39.099 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.099 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.099 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82798/exe 2024-02-05T07:52:39.100 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:39.100 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:39.100 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:39.100 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.100 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:39.100 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.100 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:39.100 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:39.100 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.100 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.100 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.100 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.100 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.100 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:39.100 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.100 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.101 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:39.101 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.101 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.101 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:39.101 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.101 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.101 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:39.101 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.101 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.101 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:39.101 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:39.101 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.101 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.101 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.101 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.101 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.102 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.102 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.102 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.102 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.102 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.102 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.102 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.102 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.103 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82876/exe 2024-02-05T07:52:39.103 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.103 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.103 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_neorados_cmd 2024-02-05T07:52:39.103 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_neorados_cmd.log 2024-02-05T07:52:39.103 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:39.103 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:39.103 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:39.103 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.103 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:39.103 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.103 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:39.103 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:39.103 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.103 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:39.103 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.104 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:39.104 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.104 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:39.104 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.104 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:39.104 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:39.104 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.104 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ cmd: /' 2024-02-05T07:52:39.104 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.104 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.104 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.104 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.104 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.104 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.104 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.104 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:39.105 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.105 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:39.105 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.105 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:39.105 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:39.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:39.105 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:39.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:39.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:39.105 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_neorados_io 2024-02-05T07:52:39.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:39.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:39.106 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:39.106 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_neorados_io.log 2024-02-05T07:52:39.106 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:39.106 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:39.106 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:39.106 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ io: /' 2024-02-05T07:52:39.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:39.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:39.106 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:39.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:39.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.108 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:39.108 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:39.108 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.108 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:39.108 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:39.108 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.108 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.108 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.108 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.108 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.108 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.108 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.108 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.108 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.108 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.109 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.110 INFO:tasks.workunit.client.0.smithi131.stdout:test misc on pid 83114 2024-02-05T07:52:39.112 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=misc 2024-02-05T07:52:39.112 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=83114 2024-02-05T07:52:39.112 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82911/exe 2024-02-05T07:52:39.112 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test misc on pid 83114' 2024-02-05T07:52:39.112 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=83114 2024-02-05T07:52:39.112 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:52:39.112 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:39.112 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.112 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.112 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s pool 2024-02-05T07:52:39.112 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.112 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.112 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.112 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.113 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.113 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.113 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.113 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.113 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.113 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' pool' 2024-02-05T07:52:39.113 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.113 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.113 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.113 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.113 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.113 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.113 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.113 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.113 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.114 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.114 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.114 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.114 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-02-05T07:52:39.114 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo pool 2024-02-05T07:52:39.114 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.114 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.114 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.114 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.114 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.114 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.114 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.114 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.114 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.114 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.114 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ ec_io: /' 2024-02-05T07:52:39.115 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:39.115 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:39.115 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.115 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:39.115 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.115 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.115 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:39.115 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.115 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:39.115 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.115 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.115 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.115 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.115 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.115 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.115 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.116 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.116 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_neorados_ec_io 2024-02-05T07:52:39.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.116 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.116 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.116 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82966/exe 2024-02-05T07:52:39.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:39.116 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.117 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.117 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.117 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.117 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.117 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.117 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.117 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.117 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.117 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.117 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.117 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=pool 2024-02-05T07:52:39.117 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=83151 2024-02-05T07:52:39.117 INFO:tasks.workunit.client.0.smithi131.stdout:test pool on pid 83151 2024-02-05T07:52:39.121 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test pool on pid 83151' 2024-02-05T07:52:39.121 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=83151 2024-02-05T07:52:39.121 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:52:39.121 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:39.121 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-02-05T07:52:39.121 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_neorados_list 2024-02-05T07:52:39.121 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s read_operations 2024-02-05T07:52:39.121 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_neorados_list.log 2024-02-05T07:52:39.122 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' read_operations' 2024-02-05T07:52:39.122 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_neorados_ec_io.log 2024-02-05T07:52:39.122 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:39.122 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/82999/exe 2024-02-05T07:52:39.122 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:39.122 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:39.122 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:39.122 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.122 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:39.122 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo read_operations 2024-02-05T07:52:39.122 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:39.122 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.122 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:39.122 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:39.122 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:39.122 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.123 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:39.123 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.123 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:39.123 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:39.123 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.123 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:39.123 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.123 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:39.123 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.123 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.123 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.123 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:39.123 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.123 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:39.123 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.123 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.124 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.124 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.124 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.124 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.124 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.124 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.124 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.124 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.124 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.124 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:39.124 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_neorados_ec_list 2024-02-05T07:52:39.124 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ list: /' 2024-02-05T07:52:39.124 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:39.124 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:39.124 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.124 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:39.125 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.125 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:39.125 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:39.125 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.125 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:39.125 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.125 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:39.125 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.125 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.125 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.125 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:39.125 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.125 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:39.125 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.126 INFO:tasks.workunit.client.0.smithi131.stdout:test read_operations on pid 83178 2024-02-05T07:52:39.129 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=read_operations 2024-02-05T07:52:39.129 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=83178 2024-02-05T07:52:39.129 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test read_operations on pid 83178' 2024-02-05T07:52:39.129 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=83178 2024-02-05T07:52:39.129 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:52:39.129 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:39.130 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-02-05T07:52:39.130 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ ec_list: /' 2024-02-05T07:52:39.130 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_neorados_ec_list.log 2024-02-05T07:52:39.130 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:39.130 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s snapshots 2024-02-05T07:52:39.130 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:39.130 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' snapshots' 2024-02-05T07:52:39.130 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:39.130 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:39.130 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.130 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:39.130 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo snapshots 2024-02-05T07:52:39.130 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:39.130 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:39.130 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:39.130 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:39.131 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.131 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:39.131 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.131 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:39.131 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:39.131 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:39.131 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.131 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:39.131 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.131 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:39.131 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:39.131 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:39.131 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.131 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:39.131 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.131 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:39.132 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.132 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:39.132 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:39.132 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:39.132 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.132 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:39.132 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.132 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:39.132 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:39.132 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.132 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:39.132 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.132 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:39.132 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.132 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.132 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.133 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:39.133 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.133 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:39.133 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.133 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=snapshots 2024-02-05T07:52:39.133 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=83195 2024-02-05T07:52:39.133 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test snapshots on pid 83195' 2024-02-05T07:52:39.133 INFO:tasks.workunit.client.0.smithi131.stdout:test snapshots on pid 83195 2024-02-05T07:52:39.135 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=83195 2024-02-05T07:52:39.135 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:52:39.136 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:39.136 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s watch_notify 2024-02-05T07:52:39.136 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' watch_notify' 2024-02-05T07:52:39.136 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-02-05T07:52:39.136 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:39.136 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:39.136 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:39.136 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.136 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:39.136 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.136 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:39.136 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:39.136 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:39.137 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.137 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:39.137 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.137 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:39.137 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:39.137 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:39.137 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.137 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:39.137 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.137 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:39.137 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.137 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:39.137 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:39.137 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:39.137 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:39.137 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:39.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:39.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo watch_notify 2024-02-05T07:52:39.138 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:39.138 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:39.138 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:39.138 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:39.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:39.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:39.138 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:39.138 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:39.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:39.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:39.139 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.140 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:39.140 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:39.140 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.140 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:39.140 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.140 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:39.140 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=watch_notify 2024-02-05T07:52:39.140 INFO:tasks.workunit.client.0.smithi131.stdout:test watch_notify on pid 83235 2024-02-05T07:52:39.144 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=83235 2024-02-05T07:52:39.144 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:39.144 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test watch_notify on pid 83235' 2024-02-05T07:52:39.144 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=83235 2024-02-05T07:52:39.144 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.144 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:39.144 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:52:39.144 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:39.144 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.144 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:39.144 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-02-05T07:52:39.144 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.144 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.144 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.145 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:39.145 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.145 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:39.145 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.145 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:39.145 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:39.145 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:39.145 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.145 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:39.145 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.145 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:39.145 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:39.145 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.145 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:39.145 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s write_operations 2024-02-05T07:52:39.146 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:39.146 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' write_operations' 2024-02-05T07:52:39.146 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:39.146 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:39.146 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.146 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:39.146 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:39.146 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:39.146 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:39.146 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.146 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:39.146 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo write_operations 2024-02-05T07:52:39.146 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.146 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:39.146 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:39.147 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.147 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:39.147 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.147 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:39.147 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.147 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.147 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.147 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:39.147 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.147 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:39.147 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-02-05T07:52:39.147 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.147 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:39.147 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:39.147 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:39.147 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:39.148 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:39.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:39.148 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:39.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:39.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:39.148 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:39.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.148 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:39.148 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:39.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:39.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:39.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:39.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:39.149 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:39.149 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:39.149 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:39.149 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:39.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:39.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:39.149 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:39.149 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:39.150 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.150 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:39.150 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.150 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:39.150 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:39.150 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:39.150 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.150 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:39.150 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.150 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:39.150 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.150 INFO:tasks.workunit.client.0.smithi131.stdout:test write_operations on pid 83267 2024-02-05T07:52:39.161 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=write_operations 2024-02-05T07:52:39.161 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:39.161 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=83267 2024-02-05T07:52:39.161 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test write_operations on pid 83267' 2024-02-05T07:52:39.161 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=83267 2024-02-05T07:52:39.161 INFO:tasks.workunit.client.0.smithi131.stderr:+ ret=0 2024-02-05T07:52:39.161 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:52:39.161 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T07:52:39.161 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82140 2024-02-05T07:52:39.161 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82140 2024-02-05T07:52:39.161 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:39.162 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:39.163 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:39.163 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:39.163 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:39.163 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.163 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:39.163 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.163 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.163 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:39.163 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:39.163 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:39.163 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:39.163 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.163 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:39.163 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.163 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.163 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:39.164 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:39.164 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:39.164 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:39.164 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:39.164 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.164 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:39.164 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.164 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:39.164 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.164 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:52:39.164 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-02-05T07:52:39.164 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-02-05T07:52:39.164 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:52:39.164 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-02-05T07:52:39.165 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:39.165 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:39.165 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:39.165 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:39.165 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:39.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:39.165 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:39.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:39.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:39.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:39.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:39.166 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:39.166 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-02-05T07:52:39.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:52:39.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-02-05T07:52:39.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:52:39.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.166 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:39.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:52:39.166 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:39.166 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:39.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:39.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:39.167 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:52:39.167 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:39.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:52:39.167 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:52:39.167 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.167 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:52:39.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:52:39.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:52:39.167 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.168 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.168 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:39.168 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.168 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:39.168 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:39.168 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.168 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.168 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.168 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.168 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.168 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.168 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.168 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.168 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.168 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.169 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.169 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.169 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.169 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.169 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.169 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.169 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.169 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:39.170 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.170 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.170 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.170 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.170 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.170 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.170 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:39.170 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:39.170 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:39.170 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.170 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.170 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:39.170 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.170 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.170 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:39.171 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.171 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.171 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:39.171 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.171 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.171 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.171 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.171 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.171 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.171 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.171 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.171 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.171 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:52:39.171 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:52:39.171 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:52:39.172 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.172 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:52:39.172 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.172 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:52:39.172 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:52:39.172 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-02-05T07:52:39.172 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.172 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:52:39.172 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.172 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:52:39.172 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-02-05T07:52:39.172 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-02-05T07:52:39.172 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.172 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:52:39.172 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.173 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:39.173 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:52:39.173 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.173 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/83114/exe 2024-02-05T07:52:39.173 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:39.173 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:39.173 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.173 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:39.173 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.173 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:39.173 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:39.173 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.173 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:39.173 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.173 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:39.173 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/83151/exe 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.174 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.175 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.175 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.175 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.175 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.175 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.175 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.175 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.175 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.175 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.175 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.175 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.175 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.175 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.175 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.175 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.175 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.176 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_neorados_misc 2024-02-05T07:52:39.176 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.176 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ misc: /' 2024-02-05T07:52:39.176 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ pool: /' 2024-02-05T07:52:39.176 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:39.176 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:39.176 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:39.176 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.176 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:39.176 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.176 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:39.176 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:39.176 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.176 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_neorados_pool 2024-02-05T07:52:39.176 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:39.176 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:52:39.177 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.177 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:39.177 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:52:39.177 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:52:39.177 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.177 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:52:39.177 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.177 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:52:39.177 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:52:39.177 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:52:39.177 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.177 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:39.177 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_neorados_pool.log 2024-02-05T07:52:39.177 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.177 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:39.177 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:39.178 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.178 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:39.178 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.178 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:39.178 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:39.178 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_neorados_misc.log 2024-02-05T07:52:39.178 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.178 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.178 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.178 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.178 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.178 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:39.178 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.178 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.178 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:39.179 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.180 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:39.180 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:39.180 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:52:39.180 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:52:39.180 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:52:39.180 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.180 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:52:39.180 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.180 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:52:39.180 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:52:39.180 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.180 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:52:39.180 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.180 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:52:39.180 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.181 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:52:39.181 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.181 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:52:39.181 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-02-05T07:52:39.181 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/83178/exe 2024-02-05T07:52:39.181 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.181 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.181 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.181 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.181 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.181 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-02-05T07:52:39.181 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.181 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.181 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.181 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.182 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.182 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.182 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.182 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.182 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.182 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.182 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/83195/exe 2024-02-05T07:52:39.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:52:39.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:52:39.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:52:39.183 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.183 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.184 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:52:39.184 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.184 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.184 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.184 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.184 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.184 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.184 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.184 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.184 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.184 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.184 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.184 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.184 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.184 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.184 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.185 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.185 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.185 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.185 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.185 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.185 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.185 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/83235/exe 2024-02-05T07:52:39.185 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_neorados_read_operations 2024-02-05T07:52:39.185 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ read_operations: /' 2024-02-05T07:52:39.185 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.185 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.185 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.185 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.185 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:52:39.186 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.186 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:52:39.186 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:52:39.186 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.186 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:52:39.186 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:52:39.186 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.186 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.186 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:52:39.186 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-02-05T07:52:39.186 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-02-05T07:52:39.186 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.186 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.186 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:52:39.186 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.187 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:52:39.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:52:39.187 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:52:39.187 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:52:39.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:52:39.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:52:39.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-02-05T07:52:39.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:52:39.187 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:52:39.187 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_neorados_snapshots.log 2024-02-05T07:52:39.187 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ snapshots: /' 2024-02-05T07:52:39.187 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/83267/exe 2024-02-05T07:52:39.187 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.188 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.188 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.188 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.188 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.188 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.188 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.188 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.188 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.188 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.188 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_neorados_watch_notify 2024-02-05T07:52:39.188 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_neorados_snapshots 2024-02-05T07:52:39.188 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_neorados_read_operations.log 2024-02-05T07:52:39.188 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-02-05T07:52:39.188 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-02-05T07:52:39.188 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ watch_notify: /' 2024-02-05T07:52:39.188 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-02-05T07:52:39.189 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-02-05T07:52:39.189 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-02-05T07:52:39.189 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-02-05T07:52:39.189 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-02-05T07:52:39.189 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-02-05T07:52:39.189 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:52:39.189 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-02-05T07:52:39.189 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:52:39.189 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-02-05T07:52:39.189 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_neorados_write_operations.log 2024-02-05T07:52:39.189 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ write_operations: /' 2024-02-05T07:52:39.189 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_neorados_write_operations 2024-02-05T07:52:39.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:39 smithi157 ceph-mon[33345]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:39.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:39 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:39.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:39 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3537436484' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-05T07:52:39.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:39 smithi131 ceph-mon[41654]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:39.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:39 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:39.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:39 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3537436484' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-05T07:52:39.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:39 smithi131 ceph-mon[29507]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:39.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:39.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3537436484' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-05T07:52:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:52:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:52:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:52:44.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:44 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:44.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2979993734' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi131-83167-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:44.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:44.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi131-83167-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:44 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2979993734' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi131-83167-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi131-83167-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:45.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:44 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:45.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2979993734' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi131-83167-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:45.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:45.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi131-83167-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:46.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/547324414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-82038-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: osdmap e75: 8 total, 8 up, 8 in 2024-02-05T07:52:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi131-83124-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3560824441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi131-82060-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi131-83167-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1308530244' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-82128-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2264131486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-82517-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3385412686' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-82377-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-82124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4192267102' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-82437-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1662796144' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi131-82071-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4080935083' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-82754-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2979993734' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi131-83167-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/962893818' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82770-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/625632808' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-82449-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3247265531' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1368508990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-82218-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1460684236' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-05T07:52:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:45 smithi157 ceph-mon[33345]: osdmap e76: 8 total, 8 up, 8 in 2024-02-05T07:52:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/547324414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-82038-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: osdmap e75: 8 total, 8 up, 8 in 2024-02-05T07:52:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi131-83124-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3560824441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi131-82060-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi131-83167-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1308530244' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-82128-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2264131486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-82517-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3385412686' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-82377-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-82124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4192267102' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-82437-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1662796144' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi131-82071-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4080935083' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-82754-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2979993734' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi131-83167-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/962893818' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82770-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/547324414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-82038-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:46.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: osdmap e75: 8 total, 8 up, 8 in 2024-02-05T07:52:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi131-83124-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3560824441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi131-82060-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi131-83167-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1308530244' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-82128-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2264131486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-82517-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3385412686' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-82377-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-82124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4192267102' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-82437-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1662796144' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi131-82071-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4080935083' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-82754-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2979993734' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi131-83167-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/962893818' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82770-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/625632808' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-82449-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3247265531' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1368508990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-82218-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1460684236' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-05T07:52:46.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[41654]: osdmap e76: 8 total, 8 up, 8 in 2024-02-05T07:52:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/625632808' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-82449-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3247265531' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1368508990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-82218-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1460684236' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-05T07:52:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:45 smithi131 ceph-mon[29507]: osdmap e76: 8 total, 8 up, 8 in 2024-02-05T07:52:47.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:47 smithi157 ceph-mon[33345]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:47.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:47 smithi157 ceph-mon[33345]: pgmap v71: 196 pgs: 64 unknown, 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:52:47.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:47 smithi157 ceph-mon[33345]: pgmap v72: 196 pgs: 44 creating+peering, 132 active+clean, 20 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:52:47.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-82038-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-82128-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-82517-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:47.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-82377-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-82124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-82437-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi131-82071-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-82754-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi131-83167-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[41654]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[41654]: pgmap v71: 196 pgs: 64 unknown, 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:52:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[41654]: pgmap v72: 196 pgs: 44 creating+peering, 132 active+clean, 20 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:52:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-82038-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-82128-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-82517-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-82377-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-82124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-82437-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi131-82071-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-82754-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi131-83167-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:47.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[29507]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:47.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[29507]: pgmap v71: 196 pgs: 64 unknown, 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:52:47.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[29507]: pgmap v72: 196 pgs: 44 creating+peering, 132 active+clean, 20 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:52:47.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-82038-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-82128-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-82517-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:47.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-82377-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-82124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-82437-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi131-82071-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-82754-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:47.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi131-83167-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:48.629 INFO:tasks.workunit.client.0.smithi131.stdout: handler_error: Running main() from gmock_main.cc 2024-02-05T07:52:48.629 INFO:tasks.workunit.client.0.smithi131.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-02-05T07:52:48.629 INFO:tasks.workunit.client.0.smithi131.stdout: handler_error: [----------] Global test environment set-up. 2024-02-05T07:52:48.630 INFO:tasks.workunit.client.0.smithi131.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-02-05T07:52:48.630 INFO:tasks.workunit.client.0.smithi131.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-02-05T07:52:48.630 INFO:tasks.workunit.client.0.smithi131.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (9589 ms) 2024-02-05T07:52:48.630 INFO:tasks.workunit.client.0.smithi131.stdout: handler_error: [----------] 1 test from neocls_handler_error (9589 ms total) 2024-02-05T07:52:48.630 INFO:tasks.workunit.client.0.smithi131.stdout: handler_error: 2024-02-05T07:52:48.630 INFO:tasks.workunit.client.0.smithi131.stdout: handler_error: [----------] Global test environment tear-down 2024-02-05T07:52:48.630 INFO:tasks.workunit.client.0.smithi131.stdout: handler_error: [==========] 1 test from 1 test suite ran. (9589 ms total) 2024-02-05T07:52:48.630 INFO:tasks.workunit.client.0.smithi131.stdout: handler_error: [ PASSED ] 1 test. 2024-02-05T07:52:48.698 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-02-05T07:52:48.698 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [----------] Global test environment set-up. 2024-02-05T07:52:48.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [----------] 4 tests from LibRadosService 2024-02-05T07:52:48.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-02-05T07:52:48.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5068 ms) 2024-02-05T07:52:48.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-02-05T07:52:48.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ OK ] LibRadosService.RegisterLate (27 ms) 2024-02-05T07:52:48.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-02-05T07:52:48.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: cluster: 2024-02-05T07:52:48.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: id: 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:52:48.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: health: HEALTH_OK 2024-02-05T07:52:48.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-02-05T07:52:48.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: services: 2024-02-05T07:52:48.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-05T07:52:48.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-02-05T07:52:48.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-02-05T07:52:48.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-02-05T07:52:48.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-05T07:52:48.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-02-05T07:52:48.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: data: 2024-02-05T07:52:48.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: pools: 6 pools, 132 pgs 2024-02-05T07:52:48.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: objects: 198 objects, 582 KiB 2024-02-05T07:52:48.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: usage: 235 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:52:48.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: pgs: 132 active+clean 2024-02-05T07:52:48.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-02-05T07:52:48.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: io: 2024-02-05T07:52:48.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: client: 1.2 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-02-05T07:52:48.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-02-05T07:52:48.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-02-05T07:52:48.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: cluster: 2024-02-05T07:52:48.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: id: 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:52:48.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: health: HEALTH_OK 2024-02-05T07:52:48.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-02-05T07:52:48.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: services: 2024-02-05T07:52:48.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-05T07:52:48.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-02-05T07:52:48.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-02-05T07:52:48.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-02-05T07:52:48.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-05T07:52:48.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-02-05T07:52:48.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: data: 2024-02-05T07:52:48.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: pools: 6 pools, 132 pgs 2024-02-05T07:52:48.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: objects: 198 objects, 582 KiB 2024-02-05T07:52:48.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: usage: 235 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:52:48.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: pgs: 132 active+clean 2024-02-05T07:52:48.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-02-05T07:52:48.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: io: 2024-02-05T07:52:48.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: client: 1.2 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-02-05T07:52:48.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-02-05T07:52:48.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-02-05T07:52:48.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: cluster: 2024-02-05T07:52:48.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: id: 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T07:52:48.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: health: HEALTH_WARN 2024-02-05T07:52:48.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 26 pool(s) do not have an application enabled 2024-02-05T07:52:48.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-02-05T07:52:48.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: services: 2024-02-05T07:52:48.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-05T07:52:48.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-02-05T07:52:48.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-02-05T07:52:48.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-02-05T07:52:48.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-02-05T07:52:48.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-05T07:52:48.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-02-05T07:52:48.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: data: 2024-02-05T07:52:48.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: pools: 41 pools, 1252 pgs 2024-02-05T07:52:48.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: objects: 198 objects, 582 KiB 2024-02-05T07:52:48.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: usage: 235 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:52:48.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: pgs: 85.942% pgs unknown 2024-02-05T07:52:48.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 3.514% pgs not active 2024-02-05T07:52:48.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 1076 unknown 2024-02-05T07:52:48.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 132 active+clean 2024-02-05T07:52:48.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:52:48.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: pgmap v74: 900 pgs: 44 creating+peering, 132 active+clean, 724 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:48.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi131-83124-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:48.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3560824441' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi131-82060-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/444388714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-82092-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3258827078' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-05T07:52:48.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: osdmap e77: 8 total, 8 up, 8 in 2024-02-05T07:52:48.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:48.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:52:48.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82770-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi131-83124-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi131-83124-1"}]: dispatch 2024-02-05T07:52:48.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-82449-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-82218-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-82092-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-82038-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-82128-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-82517-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-82377-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-82124-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-82437-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi131-82071-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-82754-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1757294297' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3289404856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-82290-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2672547644' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-82399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: pgmap v77: 1252 pgs: 44 creating+peering, 132 active+clean, 1076 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:48.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1773716977' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3562579972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82555-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: osdmap e78: 8 total, 8 up, 8 in 2024-02-05T07:52:48.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:48.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-82399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82555-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-82290-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: pgmap v74: 900 pgs: 44 creating+peering, 132 active+clean, 724 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:48.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi131-83124-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:48.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3560824441' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi131-82060-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/444388714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-82092-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3258827078' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-05T07:52:48.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: osdmap e77: 8 total, 8 up, 8 in 2024-02-05T07:52:48.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:48.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:52:48.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82770-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi131-83124-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi131-83124-1"}]: dispatch 2024-02-05T07:52:48.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-82449-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-82218-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-82092-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-82038-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-82128-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-82517-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: pgmap v74: 900 pgs: 44 creating+peering, 132 active+clean, 724 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:48.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi131-83124-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:48.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3560824441' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi131-82060-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/444388714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-82092-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3258827078' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-05T07:52:48.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: osdmap e77: 8 total, 8 up, 8 in 2024-02-05T07:52:48.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:48.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:52:48.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82770-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi131-83124-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi131-83124-1"}]: dispatch 2024-02-05T07:52:48.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-82449-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-82218-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-82092-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-82038-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-82128-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-82517-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-82377-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-82124-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-82437-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi131-82071-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-82754-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1757294297' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3289404856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-82290-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2672547644' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-82399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: pgmap v77: 1252 pgs: 44 creating+peering, 132 active+clean, 1076 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:48.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1773716977' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3562579972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82555-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: osdmap e78: 8 total, 8 up, 8 in 2024-02-05T07:52:48.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:48.773 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-82399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.773 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.773 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.773 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82555-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.773 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-82290-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.773 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-82377-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.773 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-82124-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.773 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-82437-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.773 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi131-82071-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.774 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-82754-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:48.774 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1757294297' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.774 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3289404856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-82290-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.774 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2672547644' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-82399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.774 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: pgmap v77: 1252 pgs: 44 creating+peering, 132 active+clean, 1076 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:48.774 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1773716977' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.774 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3562579972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82555-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.774 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: osdmap e78: 8 total, 8 up, 8 in 2024-02-05T07:52:48.774 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:48.774 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-82399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.774 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.774 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.774 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82555-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.775 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-82290-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:48.927 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: api_list: [==========] Running 11 tests from 3 test suites. 2024-02-05T07:52:48.927 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [----------] Global test environment set-up. 2024-02-05T07:52:48.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [----------] 7 tests from LibRadosList 2024-02-05T07:52:48.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-02-05T07:52:48.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosList.ListObjects (1317 ms) 2024-02-05T07:52:48.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-02-05T07:52:48.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (79 ms) 2024-02-05T07:52:48.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-02-05T07:52:48.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: myset foo1,foo2,foo3 2024-02-05T07:52:48.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo1 2024-02-05T07:52:48.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo2 2024-02-05T07:52:48.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo3 2024-02-05T07:52:48.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: myset foo1,foo4,foo5 2024-02-05T07:52:48.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo4 2024-02-05T07:52:48.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo5 2024-02-05T07:52:48.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo1 2024-02-05T07:52:48.930 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: myset foo6,foo7 2024-02-05T07:52:48.930 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo7 2024-02-05T07:52:48.930 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo6 2024-02-05T07:52:48.930 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-05T07:52:48.930 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns1:foo4 2024-02-05T07:52:48.930 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns1:foo5 2024-02-05T07:52:48.930 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns2:foo7 2024-02-05T07:52:48.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns2:foo6 2024-02-05T07:52:48.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns1:foo1 2024-02-05T07:52:48.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: :foo1 2024-02-05T07:52:48.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: :foo2 2024-02-05T07:52:48.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: :foo3 2024-02-05T07:52:48.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (95 ms) 2024-02-05T07:52:48.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-02-05T07:52:48.932 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 1 0 2024-02-05T07:52:48.932 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 10 0 2024-02-05T07:52:48.932 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 13 0 2024-02-05T07:52:48.932 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 7 0 2024-02-05T07:52:48.932 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 14 0 2024-02-05T07:52:48.932 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 0 0 2024-02-05T07:52:48.932 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 15 0 2024-02-05T07:52:48.932 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 11 0 2024-02-05T07:52:48.933 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 5 0 2024-02-05T07:52:48.933 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 8 0 2024-02-05T07:52:48.933 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 6 0 2024-02-05T07:52:48.933 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 3 0 2024-02-05T07:52:48.933 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 4 0 2024-02-05T07:52:48.933 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 12 0 2024-02-05T07:52:48.933 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 9 0 2024-02-05T07:52:48.933 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 2 0 2024-02-05T07:52:48.933 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-05T07:52:48.933 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (73 ms) 2024-02-05T07:52:48.933 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-02-05T07:52:48.933 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: x cursor=MIN 2024-02-05T07:52:48.934 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=1 cursor=13:02547ec2:::1:head 2024-02-05T07:52:48.934 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=10 cursor=13:52ea6a34:::10:head 2024-02-05T07:52:48.934 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=13 cursor=13:566253c9:::13:head 2024-02-05T07:52:48.934 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=7 cursor=13:5c6b0b28:::7:head 2024-02-05T07:52:48.934 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=14 cursor=13:62a1935d:::14:head 2024-02-05T07:52:48.934 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=0 cursor=13:6cac518f:::0:head 2024-02-05T07:52:48.934 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=15 cursor=13:863748b0:::15:head 2024-02-05T07:52:48.934 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=11 cursor=13:89d3ae78:::11:head 2024-02-05T07:52:48.934 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=5 cursor=13:b29083e3:::5:head 2024-02-05T07:52:48.934 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=8 cursor=13:bd63b0f1:::8:head 2024-02-05T07:52:48.934 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=6 cursor=13:c4fdafeb:::6:head 2024-02-05T07:52:48.934 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=3 cursor=13:cfc208b3:::3:head 2024-02-05T07:52:48.934 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=4 cursor=13:d83876eb:::4:head 2024-02-05T07:52:48.934 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=12 cursor=13:de5d7c5f:::12:head 2024-02-05T07:52:48.934 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=9 cursor=13:e960b815:::9:head 2024-02-05T07:52:48.934 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=2 cursor=13:f905c69b:::2:head 2024-02-05T07:52:48.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: FIRST> seek to MIN oid=1 2024-02-05T07:52:48.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=1 cursor=13:02547ec2:::1:head 2024-02-05T07:52:48.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:02547ec2:::1:head 2024-02-05T07:52:48.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:02547ec2:::1:head -> 1 2024-02-05T07:52:48.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=10 cursor=13:52ea6a34:::10:head 2024-02-05T07:52:48.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:52ea6a34:::10:head 2024-02-05T07:52:48.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:52ea6a34:::10:head -> 10 2024-02-05T07:52:48.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=13 cursor=13:566253c9:::13:head 2024-02-05T07:52:48.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:566253c9:::13:head 2024-02-05T07:52:48.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:566253c9:::13:head -> 13 2024-02-05T07:52:48.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=7 cursor=13:5c6b0b28:::7:head 2024-02-05T07:52:48.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:5c6b0b28:::7:head 2024-02-05T07:52:48.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:5c6b0b28:::7:head -> 7 2024-02-05T07:52:48.996 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=14 cursor=13:62a1935d:::14:head 2024-02-05T07:52:48.996 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:62a1935d:::14:head 2024-02-05T07:52:48.996 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:62a1935d:::14:head -> 14 2024-02-05T07:52:48.996 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=0 cursor=13:6cac518f:::0:head 2024-02-05T07:52:48.996 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:6cac518f:::0:head 2024-02-05T07:52:48.996 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:6cac518f:::0:head -> 0 2024-02-05T07:52:48.996 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=15 cursor=13:863748b0:::15:head 2024-02-05T07:52:48.996 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:863748b0:::15:head 2024-02-05T07:52:48.996 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:863748b0:::15:head -> 15 2024-02-05T07:52:48.996 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=11 cursor=13:89d3ae78:::11:head 2024-02-05T07:52:48.997 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:89d3ae78:::11:head 2024-02-05T07:52:48.997 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:89d3ae78:::11:head -> 11 2024-02-05T07:52:48.997 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=5 cursor=13:b29083e3:::5:head 2024-02-05T07:52:48.997 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:b29083e3:::5:head 2024-02-05T07:52:48.997 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:b29083e3:::5:head -> 5 2024-02-05T07:52:48.997 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=8 cursor=13:bd63b0f1:::8:head 2024-02-05T07:52:48.997 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:bd63b0f1:::8:head 2024-02-05T07:52:48.997 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:bd63b0f1:::8:head -> 8 2024-02-05T07:52:48.997 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=6 cursor=13:c4fdafeb:::6:head 2024-02-05T07:52:48.997 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:c4fdafeb:::6:head 2024-02-05T07:52:48.997 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:c4fdafeb:::6:head -> 6 2024-02-05T07:52:48.997 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=3 cursor=13:cfc208b3:::3:head 2024-02-05T07:52:48.997 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:cfc208b3:::3:head 2024-02-05T07:52:48.997 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:cfc208b3:::3:head -> 3 2024-02-05T07:52:48.997 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=4 cursor=13:d83876eb:::4:head 2024-02-05T07:52:48.998 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:d83876eb:::4:head 2024-02-05T07:52:48.998 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:d83876eb:::4:head -> 4 2024-02-05T07:52:48.998 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=12 cursor=13:de5d7c5f:::12:head 2024-02-05T07:52:48.998 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:de5d7c5f:::12:head 2024-02-05T07:52:48.998 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:de5d7c5f:::12:head -> 12 2024-02-05T07:52:48.998 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=9 cursor=13:e960b815:::9:head 2024-02-05T07:52:48.998 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:e960b815:::9:head 2024-02-05T07:52:48.998 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:e960b815:::9:head -> 9 2024-02-05T07:52:48.998 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=2 cursor=13:f905c69b:::2:head 2024-02-05T07:52:48.998 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:f905c69b:::2:head 2024-02-05T07:52:48.998 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:f905c69b:::2:head -> 2 2024-02-05T07:52:48.998 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:52ea6a34:::10:head 2024-02-05T07:52:48.998 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=13:52ea6a34:::10:head expected=13:52ea6a34:::10:head 2024-02-05T07:52:48.998 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:52ea6a34:::10:head -> 10 2024-02-05T07:52:48.998 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=10 expected=10 2024-02-05T07:52:48.998 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:02547ec2:::1:head 2024-02-05T07:52:48.999 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=13:02547ec2:::1:head expected=13:02547ec2:::1:head 2024-02-05T07:52:48.999 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:02547ec2:::1:head -> 1 2024-02-05T07:52:48.999 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=1 expected=1 2024-02-05T07:52:48.999 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:d83876eb:::4:head 2024-02-05T07:52:48.999 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=13:d83876eb:::4:head expected=13:d83876eb:::4:head 2024-02-05T07:52:48.999 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:d83876eb:::4:head -> 4 2024-02-05T07:52:48.999 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=4 expected=4 2024-02-05T07:52:48.999 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:cfc208b3:::3:head 2024-02-05T07:52:48.999 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=13:cfc208b3:::3:head expected=13:cfc208b3:::3:head 2024-02-05T07:52:48.999 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:cfc208b3:::3:head -> 3 2024-02-05T07:52:48.999 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=3 expected=3 2024-02-05T07:52:48.999 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:c4fdafeb:::6:head 2024-02-05T07:52:48.999 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=13:c4fdafeb:::6:head expected=13:c4fdafeb:::6:head 2024-02-05T07:52:48.999 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:c4fdafeb:::6:head -> 6 2024-02-05T07:52:48.999 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=6 expected=6 2024-02-05T07:52:49.000 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:6cac518f:::0:head 2024-02-05T07:52:49.000 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=13:6cac518f:::0:head expected=13:6cac518f:::0:head 2024-02-05T07:52:49.000 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:6cac518f:::0:head -> 0 2024-02-05T07:52:49.000 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=0 expected=0 2024-02-05T07:52:49.000 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:62a1935d:::14:head 2024-02-05T07:52:49.000 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=13:62a1935d:::14:head expected=13:62a1935d:::14:head 2024-02-05T07:52:49.000 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:62a1935d:::14:head -> 14 2024-02-05T07:52:49.000 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=14 expected=14 2024-02-05T07:52:49.000 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:5c6b0b28:::7:head 2024-02-05T07:52:49.000 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=13:5c6b0b28:::7:head expected=13:5c6b0b28:::7:head 2024-02-05T07:52:49.000 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:5c6b0b28:::7:head -> 7 2024-02-05T07:52:49.742 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=7 api_asio: [==========] Running 12 tests from 1 test suite. 2024-02-05T07:52:49.742 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [----------] Global test environment set-up. 2024-02-05T07:52:49.742 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [----------] 12 tests from AsioRados 2024-02-05T07:52:49.742 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-02-05T07:52:49.742 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (0 ms) 2024-02-05T07:52:49.742 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-02-05T07:52:49.743 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (1 ms) 2024-02-05T07:52:49.743 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-02-05T07:52:49.743 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (0 ms) 2024-02-05T07:52:49.743 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-02-05T07:52:49.743 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (5 ms) 2024-02-05T07:52:49.743 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-02-05T07:52:49.743 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (7 ms) 2024-02-05T07:52:49.743 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-02-05T07:52:49.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (8 ms) 2024-02-05T07:52:49.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-02-05T07:52:49.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (2 ms) 2024-02-05T07:52:49.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-02-05T07:52:49.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (0 ms) 2024-02-05T07:52:49.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-02-05T07:52:49.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (1 ms) 2024-02-05T07:52:49.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-02-05T07:52:49.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (6 ms) 2024-02-05T07:52:49.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-02-05T07:52:49.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (4 ms) 2024-02-05T07:52:49.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-02-05T07:52:49.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (4 ms) 2024-02-05T07:52:49.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [----------] 12 tests from AsioRados (38 ms total) 2024-02-05T07:52:49.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: 2024-02-05T07:52:49.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [----------] Global test environment tear-down 2024-02-05T07:52:49.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (10388 ms total) 2024-02-05T07:52:49.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ PASSED ] 12 tests. 2024-02-05T07:52:49.856 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-02-05T07:52:49.856 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-02-05T07:52:49.856 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-02-05T07:52:49.856 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-02-05T07:52:49.856 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-02-05T07:52:49.856 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (58 ms) 2024-02-05T07:52:49.856 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-02-05T07:52:49.856 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (60 ms) 2024-02-05T07:52:49.856 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-02-05T07:52:49.856 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (10831 ms) 2024-02-05T07:52:49.856 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (10949 ms total) 2024-02-05T07:52:49.857 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: 2024-02-05T07:52:49.857 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-02-05T07:52:49.857 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (10949 ms total) 2024-02-05T07:52:49.857 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-02-05T07:52:49.858 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-02-05T07:52:49.858 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-02-05T07:52:49.858 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-02-05T07:52:49.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-02-05T07:52:49.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (10877 ms) 2024-02-05T07:52:49.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (10877 ms total) 2024-02-05T07:52:49.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: 2024-02-05T07:52:49.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-02-05T07:52:49.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (10877 ms total) 2024-02-05T07:52:49.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-02-05T07:52:50.100 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_1_[82841]: starting. 2024-02-05T07:52:50.101 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_1_[82841]: creating pool ceph_test_rados_list_parallel.smithi131-82794 2024-02-05T07:52:50.101 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_1_[82841]: created object 0... 2024-02-05T07:52:50.101 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_1_[82841]: created object 25... 2024-02-05T07:52:50.101 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_1_[82841]: created object 49... 2024-02-05T07:52:50.101 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_1_[82841]: finishing. 2024-02-05T07:52:50.101 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_1_[82841]: shutting down. 2024-02-05T07:52:50.101 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_2_[82843]: starting. 2024-02-05T07:52:50.101 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_2_[82843]: listing objects. 2024-02-05T07:52:50.101 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_2_[82843]: listed object 0... 2024-02-05T07:52:50.101 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_2_[82843]: listed object 25... 2024-02-05T07:52:50.101 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_2_[82843]: saw 50 objects 2024-02-05T07:52:50.101 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_2_[82843]: shutting down. 2024-02-05T07:52:50.101 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:50.102 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_3_[84074]: starting. 2024-02-05T07:52:50.102 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_3_[84074]: creating pool ceph_test_rados_list_parallel.smithi131-82794 2024-02-05T07:52:50.102 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_3_[84074]: created object 0... 2024-02-05T07:52:50.102 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_3_[84074]: created object 25... 2024-02-05T07:52:50.102 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_3_[84074]: created object 49... 2024-02-05T07:52:50.102 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_3_[84074]: finishing. 2024-02-05T07:52:50.102 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_3_[84074]: shutting down. 2024-02-05T07:52:50.102 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:50.102 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_4_[84075]: starting. 2024-02-05T07:52:50.102 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_4_[84075]: listing objects. 2024-02-05T07:52:50.102 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_4_[84075]: listed object 0... 2024-02-05T07:52:50.102 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_4_[84075]: listed object 25... 2024-02-05T07:52:50.102 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_4_[84075]: saw 50 objects 2024-02-05T07:52:50.103 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_4_[84075]: shutting down. 2024-02-05T07:52:50.103 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:50.103 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_5_[84077]: starting. 2024-02-05T07:52:50.103 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_5_[84077]: removed 25 objects... 2024-02-05T07:52:50.103 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_5_[84077]: removed half of the objects 2024-02-05T07:52:50.103 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_5_[84077]: removed 50 objects... 2024-02-05T07:52:50.103 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_5_[84077]: removed 50 objects 2024-02-05T07:52:50.103 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_5_[84077]: shutting down. 2024-02-05T07:52:50.103 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:50.103 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:50.104 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_6_[84184]: starting. 2024-02-05T07:52:50.104 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_6_[84184]: creating pool ceph_test_rados_list_parallel.smithi131-82794 2024-02-05T07:52:50.104 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_6_[84184]: created object 0... 2024-02-05T07:52:50.104 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_6_[84184]: created object 25... 2024-02-05T07:52:50.104 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_6_[84184]: created object 49... 2024-02-05T07:52:50.104 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_6_[84184]: finishing. 2024-02-05T07:52:50.104 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_6_[84184]: shutting down. 2024-02-05T07:52:50.104 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:50.104 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:50.104 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_7_[84185]: starting. 2024-02-05T07:52:50.104 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_7_[84185]: listing objects. 2024-02-05T07:52:50.105 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_7_[84185]: listed object 0... 2024-02-05T07:52:50.105 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_7_[84185]: listed object 25... 2024-02-05T07:52:50.105 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_7_[84185]: listed object 50... 2024-02-05T07:52:50.105 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_7_[84185]: saw 56 objects 2024-02-05T07:52:50.105 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_7_[84185]: shutting down. 2024-02-05T07:52:50.105 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:50.105 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:50.105 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_8_[84186]: starting. 2024-02-05T07:52:50.105 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_8_[84186]: added 25 objects... 2024-02-05T07:52:50.105 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_8_[84186]: added half of the objects 2024-02-05T07:52:50.105 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_8_[84186]: added 50 objects... 2024-02-05T07:52:50.105 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_8_[84186]: added 50 objects 2024-02-05T07:52:50.105 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_8_[84186]: shutting down. 2024-02-05T07:52:50.105 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:50.106 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:50.955 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ********************** api_c_read_operations: Running main() from gmock_main.cc 2024-02-05T07:52:50.956 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-02-05T07:52:50.956 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-02-05T07:52:50.956 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-02-05T07:52:50.956 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-02-05T07:52:50.956 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-02-05T07:52:50.956 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-02-05T07:52:50.956 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (1249 ms) 2024-02-05T07:52:50.956 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-02-05T07:52:50.956 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (24 ms) 2024-02-05T07:52:50.956 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-02-05T07:52:50.956 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (14 ms) 2024-02-05T07:52:50.956 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-02-05T07:52:50.957 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (31 ms) 2024-02-05T07:52:50.957 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-02-05T07:52:50.957 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (9 ms) 2024-02-05T07:52:50.957 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-02-05T07:52:50.957 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (30 ms) 2024-02-05T07:52:50.957 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-02-05T07:52:50.957 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (21 ms) 2024-02-05T07:52:50.957 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-02-05T07:52:50.957 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (13 ms) 2024-02-05T07:52:50.957 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-02-05T07:52:50.957 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (6 ms) 2024-02-05T07:52:50.957 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-02-05T07:52:50.957 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (7 ms) 2024-02-05T07:52:50.957 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-02-05T07:52:50.957 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (6 ms) 2024-02-05T07:52:50.957 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-02-05T07:52:50.958 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (9 ms) 2024-02-05T07:52:50.958 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-02-05T07:52:50.958 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (17 ms) 2024-02-05T07:52:50.958 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-02-05T07:52:50.958 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (11 ms) 2024-02-05T07:52:50.958 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-02-05T07:52:50.958 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (9 ms) 2024-02-05T07:52:50.958 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-02-05T07:52:50.959 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (3 ms) 2024-02-05T07:52:50.959 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (1459 ms total) 2024-02-05T07:52:50.959 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: 2024-02-05T07:52:50.959 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-02-05T07:52:50.959 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (11994 ms total) 2024-02-05T07:52:50.959 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-02-05T07:52:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi131-83167-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi131-83167-1"}]': finished 2024-02-05T07:52:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82770-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-82449-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82598-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-82218-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-82092-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:51.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-82399-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82555-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-82290-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-82531-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:51.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: osdmap e79: 8 total, 8 up, 8 in 2024-02-05T07:52:51.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-82531-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:51.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1962418433' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-05T07:52:51.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi131-83167-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi131-83167-1"}]': finished 2024-02-05T07:52:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82770-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-82449-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82598-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-82218-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-82092-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:51.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-82399-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82555-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-82290-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-82531-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:51.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: osdmap e79: 8 total, 8 up, 8 in 2024-02-05T07:52:51.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-82531-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:51.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1962418433' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-05T07:52:51.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:51.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi131-83167-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi131-83167-1"}]': finished 2024-02-05T07:52:51.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82770-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-82449-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82598-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-82218-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-82092-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:51.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-82399-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82555-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-82290-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:51.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-82531-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:51.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: osdmap e79: 8 total, 8 up, 8 in 2024-02-05T07:52:51.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-82531-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:51.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1962418433' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-05T07:52:51.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:51.571 INFO:tasks.workunit.client.0.smithi131.stdout:********* 2024-02-05T07:52:51.571 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_9_[84359]: starting. 2024-02-05T07:52:51.571 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_9_[84359]: creating pool ceph_test_rados_list_parallel.smithi131-82794 2024-02-05T07:52:51.571 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_9_[84359]: created object 0... 2024-02-05T07:52:51.572 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_9_[84359]: created object 25... 2024-02-05T07:52:51.572 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_9_[84359]: created object 49... 2024-02-05T07:52:51.572 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_9_[84359]: finishing. 2024-02-05T07:52:51.572 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_9_[84359]: shutting down. 2024-02-05T07:52:51.572 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.572 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.572 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.572 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_10_[84360]: starting. 2024-02-05T07:52:51.572 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_10_[84360]: listing objects. 2024-02-05T07:52:51.572 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_10_[84360]: listed object 0... 2024-02-05T07:52:51.572 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_10_[84360]: listed object 25... 2024-02-05T07:52:51.572 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_10_[84360]: listed object 50... 2024-02-05T07:52:51.573 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_10_[84360]: listed object 75... 2024-02-05T07:52:51.573 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_10_[84360]: saw 93 objects 2024-02-05T07:52:51.573 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_10_[84360]: shutting down. 2024-02-05T07:52:51.573 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.573 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.573 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.573 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_13_[84363]: starting. 2024-02-05T07:52:51.573 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_13_[84363]: removed 25 objects... 2024-02-05T07:52:51.573 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_13_[84363]: removed half of the objects 2024-02-05T07:52:51.573 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_13_[84363]: removed 50 objects... 2024-02-05T07:52:51.573 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_13_[84363]: removed 50 objects 2024-02-05T07:52:51.573 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_13_[84363]: shutting down. 2024-02-05T07:52:51.573 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.573 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.574 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.574 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_11_[84361]: starting. 2024-02-05T07:52:51.574 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_11_[84361]: added 25 objects... 2024-02-05T07:52:51.574 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_11_[84361]: added half of the objects 2024-02-05T07:52:51.574 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_11_[84361]: added 50 objects... 2024-02-05T07:52:51.574 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_11_[84361]: added 50 objects 2024-02-05T07:52:51.574 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_11_[84361]: shutting down. 2024-02-05T07:52:51.574 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.574 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.574 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.574 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_12_[84362]: starting. 2024-02-05T07:52:51.574 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_12_[84362]: added 25 objects... 2024-02-05T07:52:51.574 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_12_[84362]: added half of the objects 2024-02-05T07:52:51.574 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_12_[84362]: added 50 objects... 2024-02-05T07:52:51.575 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_12_[84362]: added 50 objects 2024-02-05T07:52:51.575 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_12_[84362]: shutting down. 2024-02-05T07:52:51.575 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.575 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.575 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.575 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.575 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_14_[84513]: starting. 2024-02-05T07:52:51.575 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_14_[84513]: creating pool ceph_test_rados_list_parallel.smithi131-82794 2024-02-05T07:52:51.575 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_14_[84513]: created object 0... 2024-02-05T07:52:51.575 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_14_[84513]: created object 25... 2024-02-05T07:52:51.575 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_14_[84513]: created object 49... 2024-02-05T07:52:51.575 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_14_[84513]: finishing. 2024-02-05T07:52:51.575 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_14_[84513]: shutting down. 2024-02-05T07:52:51.575 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.575 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.576 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.576 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:51.576 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[84514]: starting. 2024-02-05T07:52:51.576 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[84514]: listing objects. 2024-02-05T07:52:51.576 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[84514]: listed object 0... 2024-02-05T07:52:52.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:52.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: pgmap v79: 1068 pgs: 561 creating+peering, 339 active+clean, 168 unknown; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 652 B/s wr, 1 op/s 2024-02-05T07:52:52.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi131-83124-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi131-83124-1"}]': finished 2024-02-05T07:52:52.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:52.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:52.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:52.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-02-05T07:52:52.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-02-05T07:52:52.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-02-05T07:52:52.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-02-05T07:52:52.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-02-05T07:52:52.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-02-05T07:52:52.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-02-05T07:52:52.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-02-05T07:52:52.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-02-05T07:52:52.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-02-05T07:52:52.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-02-05T07:52:52.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-02-05T07:52:52.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-02-05T07:52:52.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-02-05T07:52:52.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-02-05T07:52:52.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-02-05T07:52:52.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-02-05T07:52:52.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-02-05T07:52:52.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:52.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: pgmap v79: 1068 pgs: 561 creating+peering, 339 active+clean, 168 unknown; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 652 B/s wr, 1 op/s 2024-02-05T07:52:52.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi131-83124-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi131-83124-1"}]': finished 2024-02-05T07:52:52.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:52.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:52.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:52.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-02-05T07:52:52.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-02-05T07:52:52.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-02-05T07:52:52.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-02-05T07:52:52.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-02-05T07:52:52.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-02-05T07:52:52.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-02-05T07:52:52.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-02-05T07:52:52.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-02-05T07:52:52.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-02-05T07:52:52.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-02-05T07:52:52.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-02-05T07:52:52.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-02-05T07:52:52.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-02-05T07:52:52.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-02-05T07:52:52.277 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-02-05T07:52:52.277 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-02-05T07:52:52.277 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-02-05T07:52:52.278 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-02-05T07:52:52.278 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-02-05T07:52:52.278 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: osdmap e80: 8 total, 8 up, 8 in 2024-02-05T07:52:52.278 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:52:52.278 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-82531-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]': finished 2024-02-05T07:52:52.278 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2422317910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-82038-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:52.279 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1051715549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:52.279 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:52.279 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:52.279 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: 12.6 deep-scrub starts 2024-02-05T07:52:52.279 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: 12.6 deep-scrub ok 2024-02-05T07:52:52.279 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: osdmap e81: 8 total, 8 up, 8 in 2024-02-05T07:52:52.280 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:52:52.280 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:52.280 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:52.280 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:52.280 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:52.280 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-82071-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:52.280 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-82071-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:52.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-02-05T07:52:52.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-02-05T07:52:52.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: osdmap e80: 8 total, 8 up, 8 in 2024-02-05T07:52:52.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:52:52.281 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-82531-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]': finished 2024-02-05T07:52:52.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2422317910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-82038-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:52.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1051715549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:52.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:52.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:52.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: 12.6 deep-scrub starts 2024-02-05T07:52:52.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: 12.6 deep-scrub ok 2024-02-05T07:52:52.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: osdmap e81: 8 total, 8 up, 8 in 2024-02-05T07:52:52.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:52:52.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:52.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:52.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:52.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:52.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-82071-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:52.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-82071-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:52.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: pgmap v79: 1068 pgs: 561 creating+peering, 339 active+clean, 168 unknown; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 652 B/s wr, 1 op/s 2024-02-05T07:52:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi131-83124-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi131-83124-1"}]': finished 2024-02-05T07:52:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-02-05T07:52:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-02-05T07:52:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-02-05T07:52:52.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-02-05T07:52:52.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-02-05T07:52:52.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-02-05T07:52:52.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-02-05T07:52:52.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-02-05T07:52:52.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-02-05T07:52:52.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-02-05T07:52:52.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-02-05T07:52:52.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-02-05T07:52:52.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-02-05T07:52:52.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-02-05T07:52:52.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-02-05T07:52:52.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-02-05T07:52:52.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-02-05T07:52:52.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-02-05T07:52:52.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-02-05T07:52:52.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-02-05T07:52:52.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: osdmap e80: 8 total, 8 up, 8 in 2024-02-05T07:52:52.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:52:52.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-82531-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]': finished 2024-02-05T07:52:52.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2422317910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-82038-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:52.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1051715549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:52.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:52.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:52.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: 12.6 deep-scrub starts 2024-02-05T07:52:52.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: 12.6 deep-scrub ok 2024-02-05T07:52:52.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: osdmap e81: 8 total, 8 up, 8 in 2024-02-05T07:52:52.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:52:52.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:52.474 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:52.475 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:52.475 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:52.475 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-82071-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:52.475 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-82071-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:52.532 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[84514]: listed object 2 api_cmd: Running main() from gmock_main.cc 2024-02-05T07:52:52.532 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-02-05T07:52:52.532 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [----------] Global test environment set-up. 2024-02-05T07:52:52.532 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-02-05T07:52:52.532 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-02-05T07:52:52.532 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (55 ms) 2024-02-05T07:52:52.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-02-05T07:52:52.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (416 ms) 2024-02-05T07:52:52.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-02-05T07:52:52.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (12661 ms) 2024-02-05T07:52:52.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-02-05T07:52:52.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:49.521468+0000 mon.a [INF] from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi131-83124-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi131-83124-1"}]': finished 2024-02-05T07:52:52.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:49.895930+0000 mon.b [INF] from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:52.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:49.898587+0000 mon.b [INF] from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:52.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:50.880813+0000 mon.c [INF] from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-02-05T07:52:52.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:50.881375+0000 mon.c [INF] from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-02-05T07:52:52.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:50.881801+0000 mon.c [INF] from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-02-05T07:52:52.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:50.882178+0000 mon.c [INF] from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-02-05T07:52:52.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:50.882562+0000 mon.c [INF] from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-02-05T07:52:52.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:50.882939+0000 mon.c [INF] from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-02-05T07:52:52.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:50.883295+0000 mon.c [INF] from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-02-05T07:52:52.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:50.883682+0000 mon.c [INF] from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-02-05T07:52:52.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:50.884074+0000 mon.c [INF] from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-02-05T07:52:52.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:50.884481+0000 mon.c [INF] from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-02-05T07:52:52.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:50.941293+0000 mon.a [INF] from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:52:53.008 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:50.942012+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-82531-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]': finished 2024-02-05T07:52:53.008 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:50.949625+0000 mon.b [INF] from='client.? 172.21.15.131:0/2422317910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-82038-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:53.008 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:50.950184+0000 mon.b [INF] from='client.? 172.21.15.131:0/1051715549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:53.008 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:51.013293+0000 mon.c [INF] from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:53.009 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:51.467084+0000 mon.a [INF] from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:52:53.009 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:51.467892+0000 mon.a [INF] from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:53.009 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:51.740113+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:53.009 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:51.740767+0000 mon.c [INF] from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:53.009 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:51.742985+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:53.009 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:51.743516+0000 mon.c [INF] from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-82071-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:53.009 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:51.744060+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-82071-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:53.009 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:51.994748+0000 mon.a [INF] from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:53.009 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:51.994800+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-82071-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:53.009 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:52.010578+0000 mon.c [INF] from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-82071-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:53.009 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:52.011842+0000 mon.c [INF] from='client.? 172.21.15.131:0/2979993734' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:53.024 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:52.012961+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-85... 2024-02-05T07:52:53.024 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[84514]: listed object 50... 2024-02-05T07:52:53.024 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[84514]: listed object 75... 2024-02-05T07:52:53.025 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[84514]: listed object 100... 2024-02-05T07:52:53.025 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[84514]: listed object 125... 2024-02-05T07:52:53.025 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[84514]: saw 150 objects 2024-02-05T07:52:53.025 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[84514]: shutting down. 2024-02-05T07:52:53.025 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:53.025 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:53.025 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:53.025 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:53.025 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_16_[84515]: starting. 2024-02-05T07:52:53.025 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_16_[84515]: added 25 objects... 2024-02-05T07:52:53.025 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_16_[84515]: added half of the objects 2024-02-05T07:52:53.025 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_16_[84515]: added 50 objects... 2024-02-05T07:52:53.025 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_16_[84515]: added 50 objects 2024-02-05T07:52:53.025 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_16_[84515]: shutting down. 2024-02-05T07:52:53.025 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:53.025 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:53.026 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:53.026 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:53.026 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-02-05T07:52:53.026 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******* SUCCESS ********** 2024-02-05T07:52:53.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:52:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:52:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:52:53.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: pgmap v82: 828 pgs: 194 creating+peering, 226 active+clean, 408 unknown; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:52:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-82071-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: osdmap e82: 8 total, 8 up, 8 in 2024-02-05T07:52:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-82071-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2979993734' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-82071-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-82092-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:52:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi131-83124-1"}]: dispatch 2024-02-05T07:52:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:53.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/784167816' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-05T07:52:53.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:53.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:52:53.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:53.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: pgmap v82: 828 pgs: 194 creating+peering, 226 active+clean, 408 unknown; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:52:53.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:53.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-82071-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:53.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: osdmap e82: 8 total, 8 up, 8 in 2024-02-05T07:52:53.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-82071-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:53.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2979993734' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:53.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-82071-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:53.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-82092-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:52:53.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:53.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:53.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi131-83124-1"}]: dispatch 2024-02-05T07:52:53.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:53.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:53.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:53.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/784167816' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-05T07:52:53.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:53.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:52:53.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: pgmap v82: 828 pgs: 194 creating+peering, 226 active+clean, 408 unknown; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:52:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-82071-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: osdmap e82: 8 total, 8 up, 8 in 2024-02-05T07:52:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-82071-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2979993734' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-82071-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-82092-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:52:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi131-83124-1"}]: dispatch 2024-02-05T07:52:53.374 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:53.374 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:53.374 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:53.374 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/784167816' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-05T07:52:53.374 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:53.374 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:52:53.374 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:54.024 INFO:tasks.workunit.client.0.smithi131.stdout:2071-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:54.024 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:52.016998+0000 mon.a [INF] from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-82092-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:52:54.024 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:52.292502+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:54.024 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:52.292720+0000 mon.a [INF] from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi131-83124-1"}]: dispatch 2024-02-05T07:52:54.024 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:52.292874+0000 mon.a [INF] from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:54.024 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:52.527149+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:54.024 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:52.528128+0000 mon.b [INF] from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:54.024 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:52.531525+0000 mon.c [INF] from='client.? 172.21.15.131:0/784167816' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-05T07:52:54.024 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:52.766097+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:54.025 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:52.766414+0000 mon.a [INF] from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:52:54.025 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:52.766911+0000 mon.b [INF] from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:54.025 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:53.004830+0000 mon.a [INF] from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:52:54.025 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:53.011440+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-82038-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:54.025 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:53.011703+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:54.025 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:53.014285+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi131-83167-1"}]': finished 2024-02-05T07:52:54.025 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:53.014335+0000 mon.a [INF] from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi131-83124-1"}]': finished 2024-02-05T07:52:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:54 smithi157 ceph-mon[33345]: 12.5 deep-scrub starts 2024-02-05T07:52:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:54 smithi157 ceph-mon[33345]: 12.5 deep-scrub ok 2024-02-05T07:52:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:54 smithi157 ceph-mon[33345]: 12.3 deep-scrub starts 2024-02-05T07:52:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:54 smithi157 ceph-mon[33345]: 12.3 deep-scrub ok 2024-02-05T07:52:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:52:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-82038-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi131-83167-1"}]': finished 2024-02-05T07:52:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi131-83124-1"}]': finished 2024-02-05T07:52:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2979993734' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2821854274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:54 smithi157 ceph-mon[33345]: osdmap e83: 8 total, 8 up, 8 in 2024-02-05T07:52:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi131-83124-1"}]: dispatch 2024-02-05T07:52:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[41654]: 12.5 deep-scrub starts 2024-02-05T07:52:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[41654]: 12.5 deep-scrub ok 2024-02-05T07:52:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[41654]: 12.3 deep-scrub starts 2024-02-05T07:52:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[41654]: 12.3 deep-scrub ok 2024-02-05T07:52:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:52:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-82038-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi131-83167-1"}]': finished 2024-02-05T07:52:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi131-83124-1"}]': finished 2024-02-05T07:52:54.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:54.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2979993734' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:54.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:54.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2821854274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:54.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[41654]: osdmap e83: 8 total, 8 up, 8 in 2024-02-05T07:52:54.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi131-83124-1"}]: dispatch 2024-02-05T07:52:54.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:54.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[29507]: 12.5 deep-scrub starts 2024-02-05T07:52:54.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[29507]: 12.5 deep-scrub ok 2024-02-05T07:52:54.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[29507]: 12.3 deep-scrub starts 2024-02-05T07:52:54.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[29507]: 12.3 deep-scrub ok 2024-02-05T07:52:54.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:52:54.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-82038-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:54.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:54.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi131-83167-1"}]': finished 2024-02-05T07:52:54.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi131-83124-1"}]': finished 2024-02-05T07:52:54.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:54.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2979993734' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:54.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:54.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2821854274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:54.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[29507]: osdmap e83: 8 total, 8 up, 8 in 2024-02-05T07:52:54.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi131-83124-1"}]: dispatch 2024-02-05T07:52:54.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:55.045 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:53.014398+0000 mon.a [INF] from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:55.045 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:53.020721+0000 mon.c [INF] from='client.? 172.21.15.131:0/2979993734' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:55.045 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:53.530128+0000 mon.b [INF] from='client.? 172.21.15.131:0/2821854274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:55.045 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:53.772743+0000 mon.a [INF] from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi131-83124-1"}]: dispatch 2024-02-05T07:52:55.045 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:53.773002+0000 mon.a [INF] from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:55.045 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:52.531710+0000 client.admin [INF] onexx 2024-02-05T07:52:55.046 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.027144+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-05T07:52:55.046 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.027441+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:55.046 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.027634+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-82071-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-82071-16"}]': finished 2024-02-05T07:52:55.046 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.027741+0000 mon.a [INF] from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-82092-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-82092-22"}]': finished 2024-02-05T07:52:55.046 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.027821+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-82038-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:55.046 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.027909+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:55.046 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.027974+0000 mon.a [INF] from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi131-83124-1"}]': finished 2024-02-05T07:52:55.046 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.028031+0000 mon.a [INF] from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:55.046 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.036904+0000 mon.b [INF] from='client.? 172.21.15.131:0/1666130236' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:55.046 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.037713+0000 mon.a [INF] from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-82218-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:52:55.338 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.042758+0000 mon.a [INF] from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"pre open_pools_parallel: process_1_[82851]: starting. 2024-02-05T07:52:55.338 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_1_[82851]: creating pool ceph_test_rados_open_pools_parallel.smithi131-82803 2024-02-05T07:52:55.338 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_1_[82851]: created object 0... 2024-02-05T07:52:55.338 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_1_[82851]: created object 25... 2024-02-05T07:52:55.338 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_1_[82851]: created object 49... 2024-02-05T07:52:55.338 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_1_[82851]: finishing. 2024-02-05T07:52:55.338 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_1_[82851]: shutting down. 2024-02-05T07:52:55.338 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_2_[82852]: starting. 2024-02-05T07:52:55.339 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_2_[82852]: rados_pool_create. 2024-02-05T07:52:55.339 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_2_[82852]: rados_ioctx_create. 2024-02-05T07:52:55.339 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_2_[82852]: shutting down. 2024-02-05T07:52:55.339 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: ******************************* 2024-02-05T07:52:55.339 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_3_[84557]: starting. 2024-02-05T07:52:55.339 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_3_[84557]: creating pool ceph_test_rados_open_pools_parallel.smithi131-82803 2024-02-05T07:52:55.339 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_3_[84557]: created object 0... 2024-02-05T07:52:55.339 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_3_[84557]: created object 25... 2024-02-05T07:52:55.339 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_3_[84557]: created object 49... 2024-02-05T07:52:55.339 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_3_[84557]: finishing. 2024-02-05T07:52:55.339 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_3_[84557]: shutting down. 2024-02-05T07:52:55.339 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: ******************************* 2024-02-05T07:52:55.339 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_4_[84558]: starting. 2024-02-05T07:52:55.339 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_4_[84558]: rados_pool_create. 2024-02-05T07:52:55.339 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_4_[84558]: rados_ioctx_create. 2024-02-05T07:52:55.339 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_4_[84558]: shutting down. 2024-02-05T07:52:55.340 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: ******************************* 2024-02-05T07:52:55.340 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: ******************************* 2024-02-05T07:52:55.340 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-02-05T07:52:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: 12.0 deep-scrub starts 2024-02-05T07:52:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: 12.0 deep-scrub ok 2024-02-05T07:52:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: 12.8 deep-scrub starts 2024-02-05T07:52:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: 12.8 deep-scrub ok 2024-02-05T07:52:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: onexx 2024-02-05T07:52:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: 12.2 deep-scrub starts 2024-02-05T07:52:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: 12.2 deep-scrub ok 2024-02-05T07:52:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: pgmap v85: 908 pgs: 204 creating+peering, 655 active+clean, 49 unknown; 145 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 558 op/s 2024-02-05T07:52:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-05T07:52:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-82071-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-82071-16"}]': finished 2024-02-05T07:52:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-82092-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-82092-22"}]': finished 2024-02-05T07:52:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-82038-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi131-83124-1"}]': finished 2024-02-05T07:52:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: osdmap e84: 8 total, 8 up, 8 in 2024-02-05T07:52:55.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1666130236' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:55.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-82218-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:52:55.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:55.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:55.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4131594549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi131-83124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:55.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:52:55.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:55.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:52:55.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:55.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-82128-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:55.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-82399-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:55.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:55.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:55.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:55.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: 12.0 deep-scrub starts 2024-02-05T07:52:55.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: 12.0 deep-scrub ok 2024-02-05T07:52:55.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: 12.8 deep-scrub starts 2024-02-05T07:52:55.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: 12.8 deep-scrub ok 2024-02-05T07:52:55.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: onexx 2024-02-05T07:52:55.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: 12.2 deep-scrub starts 2024-02-05T07:52:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: 12.2 deep-scrub ok 2024-02-05T07:52:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: pgmap v85: 908 pgs: 204 creating+peering, 655 active+clean, 49 unknown; 145 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 558 op/s 2024-02-05T07:52:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-05T07:52:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-82071-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-82071-16"}]': finished 2024-02-05T07:52:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-82092-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-82092-22"}]': finished 2024-02-05T07:52:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-82038-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi131-83124-1"}]': finished 2024-02-05T07:52:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: osdmap e84: 8 total, 8 up, 8 in 2024-02-05T07:52:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1666130236' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-82218-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:52:55.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:55.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:55.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4131594549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi131-83124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:55.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:52:55.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: 12.0 deep-scrub starts 2024-02-05T07:52:55.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: 12.0 deep-scrub ok 2024-02-05T07:52:55.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: 12.8 deep-scrub starts 2024-02-05T07:52:55.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: 12.8 deep-scrub ok 2024-02-05T07:52:55.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: onexx 2024-02-05T07:52:55.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: 12.2 deep-scrub starts 2024-02-05T07:52:55.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: 12.2 deep-scrub ok 2024-02-05T07:52:55.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: pgmap v85: 908 pgs: 204 creating+peering, 655 active+clean, 49 unknown; 145 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 558 op/s 2024-02-05T07:52:55.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-05T07:52:55.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi131-83167-1"}]: dispatch 2024-02-05T07:52:55.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-82071-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-82071-16"}]': finished 2024-02-05T07:52:55.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-82092-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-82092-22"}]': finished 2024-02-05T07:52:55.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-82038-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:55.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:55.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3228657706' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi131-83124-1"}]': finished 2024-02-05T07:52:55.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:55.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: osdmap e84: 8 total, 8 up, 8 in 2024-02-05T07:52:55.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1666130236' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:55.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-82218-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:52:55.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:55.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:55.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4131594549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi131-83124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:55.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:52:55.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:55.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:52:55.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:55.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-82128-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:55.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-82399-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:55.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:55.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:55.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:55.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:55.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:52:55.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:55.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-82128-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:55.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-82399-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:55.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:55.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:55.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:56.268 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: Running main() from gmock_main.cc 2024-02-05T07:52:56.268 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-02-05T07:52:56.268 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [----------] Global test environment set-up. 2024-02-05T07:52:56.268 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-02-05T07:52:56.268 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: seed 82092 2024-02-05T07:52:56.268 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-02-05T07:52:56.268 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-02-05T07:52:56.268 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-02-05T07:52:56.268 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (256 ms) 2024-02-05T07:52:56.268 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-02-05T07:52:56.269 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (16 ms) 2024-02-05T07:52:56.269 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-02-05T07:52:56.269 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (3 ms) 2024-02-05T07:52:56.269 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-02-05T07:52:56.269 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (3 ms) 2024-02-05T07:52:56.269 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-02-05T07:52:56.269 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (5 ms) 2024-02-05T07:52:56.269 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-02-05T07:52:56.269 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (6 ms) 2024-02-05T07:52:56.269 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-02-05T07:52:56.269 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (3 ms) 2024-02-05T07:52:56.269 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-02-05T07:52:56.269 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (6 ms) 2024-02-05T07:52:56.269 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-02-05T07:52:56.269 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (4 ms) 2024-02-05T07:52:56.270 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-02-05T07:52:56.270 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (3 ms) 2024-02-05T07:52:56.270 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-02-05T07:52:56.270 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (6 ms) 2024-02-05T07:52:56.270 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-02-05T07:52:56.270 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (5 ms) 2024-02-05T07:52:56.270 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-02-05T07:52:56.270 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (24 ms) 2024-02-05T07:52:56.270 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-02-05T07:52:56.270 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (29 ms) 2024-02-05T07:52:56.270 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-02-05T07:52:56.270 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (22 ms) 2024-02-05T07:52:56.270 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-05T07:52:56.270 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (9 ms) 2024-02-05T07:52:56.270 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-02-05T07:52:56.270 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (6 ms) 2024-02-05T07:52:56.271 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-02-05T07:52:56.271 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (5 ms) 2024-02-05T07:52:56.271 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-02-05T07:52:56.271 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (4 ms) 2024-02-05T07:52:56.271 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (415 ms total) 2024-02-05T07:52:56.271 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: 2024-02-05T07:52:56.271 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-02-05T07:52:56.271 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-02-05T07:52:56.271 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (2187 ms) 2024-02-05T07:52:56.271 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-02-05T07:52:56.271 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (18 ms) 2024-02-05T07:52:56.271 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-02-05T07:52:56.271 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (5 ms) 2024-02-05T07:52:56.271 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-02-05T07:52:56.272 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (4 ms) 2024-02-05T07:52:56.272 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-02-05T07:52:56.272 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (3 ms) 2024-02-05T07:52:56.397 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN cls: Running main() from gmock_main.cc 2024-02-05T07:52:56.398 INFO:tasks.workunit.client.0.smithi131.stdout: cls: [==========] Running 2 tests from 1 test suite. 2024-02-05T07:52:56.398 INFO:tasks.workunit.client.0.smithi131.stdout: cls: [----------] Global test environment set-up. 2024-02-05T07:52:56.398 INFO:tasks.workunit.client.0.smithi131.stdout: cls: [----------] 2 tests from NeoRadosCls 2024-02-05T07:52:56.398 INFO:tasks.workunit.client.0.smithi131.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-02-05T07:52:56.398 INFO:tasks.workunit.client.0.smithi131.stdout: cls: [ OK ] NeoRadosCls.DNE (10814 ms) 2024-02-05T07:52:56.398 INFO:tasks.workunit.client.0.smithi131.stdout: cls: [ RUN ] NeoRadosCls.RemoteReads 2024-02-05T07:52:56.398 INFO:tasks.workunit.client.0.smithi131.stdout: cls: [ OK ] NeoRadosCls.RemoteReads (6543 ms) 2024-02-05T07:52:56.398 INFO:tasks.workunit.client.0.smithi131.stdout: cls: [----------] 2 tests from NeoRadosCls (17357 ms total) 2024-02-05T07:52:56.398 INFO:tasks.workunit.client.0.smithi131.stdout: cls: 2024-02-05T07:52:56.398 INFO:tasks.workunit.client.0.smithi131.stdout: cls: [----------] Global test environment tear-down 2024-02-05T07:52:56.398 INFO:tasks.workunit.client.0.smithi131.stdout: cls: [==========] 2 tests from 1 test suite ran. (17357 ms total) 2024-02-05T07:52:56.398 INFO:tasks.workunit.client.0.smithi131.stdout: cls: [ PASSED ] 2 tests. 2024-02-05T07:52:56.403 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_1_[82983]: starting. 2024-02-05T07:52:56.403 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_1_[82983]: creating pool ceph_test_rados_delete_pools_parallel.smithi131-82888 2024-02-05T07:52:56.403 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_1_[82983]: created object 0... 2024-02-05T07:52:56.403 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_1_[82983]: created object 25... 2024-02-05T07:52:56.403 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_1_[82983]: created object 49... 2024-02-05T07:52:56.403 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_1_[82983]: finishing. 2024-02-05T07:52:56.403 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_1_[82983]: shutting down. 2024-02-05T07:52:56.403 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_2_[82988]: starting. 2024-02-05T07:52:56.404 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_2_[82988]: deleting pool ceph_test_rados_delete_pools_parallel.smithi131-82888 2024-02-05T07:52:56.404 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_2_[82988]: shutting down. 2024-02-05T07:52:56.404 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: ******************************* 2024-02-05T07:52:56.404 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_3_[84451]: starting. 2024-02-05T07:52:56.404 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_3_[84451]: creating pool ceph_test_rados_delete_pools_parallel.smithi131-82888 2024-02-05T07:52:56.404 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_3_[84451]: created object 0... 2024-02-05T07:52:56.404 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_3_[84451]: created object 25... 2024-02-05T07:52:56.404 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_3_[84451]: created object 49... 2024-02-05T07:52:56.404 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_3_[84451]: finishing. 2024-02-05T07:52:56.404 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_3_[84451]: shutting down. 2024-02-05T07:52:56.404 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: ******************************* 2024-02-05T07:52:56.404 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_5_[84453]: starting. 2024-02-05T07:52:56.404 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_5_[84453]: listing objects. 2024-02-05T07:52:56.405 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_5_[84453]: listed object 0... 2024-02-05T07:52:56.405 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_5_[84453]: listed object 25... 2024-02-05T07:52:56.405 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_5_[84453]: saw 50 objects 2024-02-05T07:52:56.405 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_5_[84453]: shutting down. 2024-02-05T07:52:56.405 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: ******************************* 2024-02-05T07:52:56.405 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_4_[84452]: starting. 2024-02-05T07:52:56.405 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_4_[84452]: deleting pool ceph_test_rados_delete_pools_parallel.smithi131-82888 2024-02-05T07:52:56.405 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_4_[84452]: shutting down. 2024-02-05T07:52:56.405 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: ******************************* 2024-02-05T07:52:56.405 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: ******************************* 2024-02-05T07:52:56.405 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-02-05T07:52:56.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: Health check update: 17 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:52:56.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-05T07:52:56.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/784167816' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-05T07:52:56.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: twoxx 2024-02-05T07:52:56.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-05T07:52:56.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi131-83167-1"}]': finished 2024-02-05T07:52:56.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:56.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-82128-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:56.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:56.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:56.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:56.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1090436836' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi131-83167-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:56.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:56.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1051715549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:56.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: osdmap e85: 8 total, 8 up, 8 in 2024-02-05T07:52:56.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi131-83124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:56.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:56.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-82128-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:52:56.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi131-83167-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:56.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-4", "tierpool": "test-rados-api-smithi131-82290-4-cache"}]: dispatch 2024-02-05T07:52:56.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-82399-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:56.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:56.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:56.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: Health check update: 17 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:52:56.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-05T07:52:56.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/784167816' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-05T07:52:56.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: twoxx 2024-02-05T07:52:56.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-05T07:52:56.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi131-83167-1"}]': finished 2024-02-05T07:52:56.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:56.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-82128-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:56.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:56.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:56.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:56.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1090436836' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi131-83167-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:56.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:56.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1051715549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:56.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: osdmap e85: 8 total, 8 up, 8 in 2024-02-05T07:52:56.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi131-83124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:56.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:56.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-82128-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:52:56.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi131-83167-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:56.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-4", "tierpool": "test-rados-api-smithi131-82290-4-cache"}]: dispatch 2024-02-05T07:52:56.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: Health check update: 17 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:52:56.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-05T07:52:56.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/784167816' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-05T07:52:56.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: twoxx 2024-02-05T07:52:56.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-05T07:52:56.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi131-83167-1"}]': finished 2024-02-05T07:52:56.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:56.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-82128-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:56.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:56.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:56.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:56.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1090436836' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi131-83167-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:56.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:56.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1051715549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:56.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: osdmap e85: 8 total, 8 up, 8 in 2024-02-05T07:52:56.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi131-83124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:56.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:56.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-82128-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:52:56.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi131-83167-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:56.771 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-4", "tierpool": "test-rados-api-smithi131-82290-4-cache"}]: dispatch 2024-02-05T07:52:56.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-82399-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:56.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:56.772 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:56.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-82399-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:56.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:56.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:58.157 INFO:tasks.workunit.client.0.smithi131.stdout: cmd: Running main() from gmock_main.cc 2024-02-05T07:52:58.158 INFO:tasks.workunit.client.0.smithi131.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-02-05T07:52:58.158 INFO:tasks.workunit.client.0.smithi131.stdout: cmd: [----------] Global test environment set-up. 2024-02-05T07:52:58.158 INFO:tasks.workunit.client.0.smithi131.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-02-05T07:52:58.158 INFO:tasks.workunit.client.0.smithi131.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-02-05T07:52:58.158 INFO:tasks.workunit.client.0.smithi131.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (9563 ms) 2024-02-05T07:52:58.158 INFO:tasks.workunit.client.0.smithi131.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-02-05T07:52:58.158 INFO:tasks.workunit.client.0.smithi131.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (5408 ms) 2024-02-05T07:52:58.158 INFO:tasks.workunit.client.0.smithi131.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-02-05T07:52:58.158 INFO:tasks.workunit.client.0.smithi131.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (4115 ms) 2024-02-05T07:52:58.158 INFO:tasks.workunit.client.0.smithi131.stdout: cmd: [----------] 3 tests from NeoRadosCmd (19086 ms total) 2024-02-05T07:52:58.158 INFO:tasks.workunit.client.0.smithi131.stdout: cmd: 2024-02-05T07:52:58.158 INFO:tasks.workunit.client.0.smithi131.stdout: cmd: [----------] Global test environment tear-down 2024-02-05T07:52:58.159 INFO:tasks.workunit.client.0.smithi131.stdout: cmd: [==========] 3 tests from 1 test suite ran. (19086 ms total) 2024-02-05T07:52:58.159 INFO:tasks.workunit.client.0.smithi131.stdout: cmd: [ PASSED ] 3 tests. 2024-02-05T07:52:58.413 INFO:tasks.workunit.client.0.smithi131.stdout:fix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:58.413 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.105689+0000 mon.b [INF] from='client.? 172.21.15.131:0/4131594549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi131-83124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:58.413 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.799517+0000 mon.a [INF] from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:52:58.413 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.799924+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:58.413 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.800078+0000 mon.a [INF] from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:52:58.413 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.800315+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:58.414 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.800560+0000 mon.a [INF] from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-82128-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:58.414 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.800597+0000 mon.b [INF] from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-82399-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:58.414 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.800799+0000 mon.b [INF] from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:58.414 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.801141+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:58.414 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:54.801360+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:58.414 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.042459+0000 mon.a [WRN] Health check update: 17 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:52:58.414 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.043197+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-05T07:52:58.414 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.047606+0000 mon.c [INF] from='client.? 172.21.15.131:0/784167816' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-05T07:52:58.414 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.047750+0000 client.admin [INF] twoxx 2024-02-05T07:52:58.414 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.048694+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-05T07:52:58.414 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.049612+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi131-83167-1"}]': finished 2024-02-05T07:52:58.414 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.049757+0000 mon.a [INF] from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:58.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:52:58.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.049827+0000 mon.a [INF] from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-82128-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:58.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.049892+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:58.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.049968+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:58.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.058106+0000 mon.c [INF] from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:58.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.101160+0000 mon.c [INF] from='client.? 172.21.15.131:0/1090436836' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi131-83167-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:58.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.334731+0000 mon.b [INF] from='client.? 172.21.15.131:0/1051715549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:58.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.584815+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi131-83124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:58.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.585794+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:58.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.586647+0000 mon.a [INF] from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-82128-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:52:58.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.587507+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi131-83167-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:58.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.589769+0000 mon.a [INF] from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-4", "tierpool": "test-rados-api-smithi131-82290-4-cache"}]: dispatch 2024-02-05T07:52:58.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.846544+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-82399-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:58.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.846911+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:52:58.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:55.852701+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:58.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.096897+0000 mon.a [INF] from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-82218-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-82218-10"}]': finished 2024-02-05T07:52:58.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.096979+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi131-83124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:58.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.097047+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]': finished 2024-02-05T07:52:58.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.097105+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi131-83167-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:58.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.097188+0000 mon.a [INF] from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-4", "tierpool": "test-rados-api-smithi131-82290-4-cache"}]': finished 2024-02-05T07:52:58.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.097238+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-82399-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:58.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.097291+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:58.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.097368+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:58.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.101849+0000 mon.c [INF] from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:58.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.102239+0000 mon.c [INF] from='client.? 172.21.15.131:0/1090436836' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi131-83167-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:52:58.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.103014+0000 mon.b [INF] from='client.? 172.21.15.131:0/4131594549' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi131-83124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:52:58.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.103265+0000 mon.b [INF] from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-82399-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:58.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.103895+0000 mon.b [INF] from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-82449-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:58.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.107989+0000 mon.b [INF] from='client.? 172.21.15.131:0/1051715549' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-82060-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-05T07:52:59.153 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.399090+0 watch_notify: Running main() from gmock_main.cc 2024-02-05T07:52:59.154 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-02-05T07:52:59.154 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: [----------] Global test environment set-up. 2024-02-05T07:52:59.154 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-02-05T07:52:59.154 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-02-05T07:52:59.154 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: handle_notify cookie 93991163189440 notify_id 339302416385 notifier_gid 24827 2024-02-05T07:52:59.154 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (10715 ms) 2024-02-05T07:52:59.154 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-02-05T07:52:59.154 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: Trying... 2024-02-05T07:52:59.154 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: handle_notify cookie 93991175885456 notify_id 356482285568 notifier_gid 15084 2024-02-05T07:52:59.154 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: Waiting for 3.000000000s 2024-02-05T07:52:59.154 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: Timed out. 2024-02-05T07:52:59.154 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: Flushing... 2024-02-05T07:52:59.154 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: Flushed... 2024-02-05T07:52:59.154 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (9295 ms) 2024-02-05T07:52:59.155 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (20010 ms total) 2024-02-05T07:52:59.155 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: 2024-02-05T07:52:59.155 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: [----------] Global test environment tear-down 2024-02-05T07:52:59.155 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (20010 ms total) 2024-02-05T07:52:59.155 INFO:tasks.workunit.client.0.smithi131.stdout: watch_notify: [ PASSED ] 2 tests. 2024-02-05T07:52:59.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: pgmap v88: 916 pgs: 153 creating+peering, 506 active+clean, 257 unknown; 145 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-02-05T07:52:59.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-82218-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-82218-10"}]': finished 2024-02-05T07:52:59.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi131-83124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:59.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]': finished 2024-02-05T07:52:59.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi131-83167-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:59.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-4", "tierpool": "test-rados-api-smithi131-82290-4-cache"}]': finished 2024-02-05T07:52:59.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-82399-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:59.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:59.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:59.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:59.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1090436836' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi131-83167-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:52:59.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4131594549' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi131-83124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:52:59.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-82399-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:59.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-82449-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:59.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1051715549' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-82060-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-05T07:52:59.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:59.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: osdmap e86: 8 total, 8 up, 8 in 2024-02-05T07:52:59.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-05T07:52:59.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi131-83124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:52:59.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:59.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-4", "overlaypool": "test-rados-api-smithi131-82290-4-cache"}]: dispatch 2024-02-05T07:52:59.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-82399-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:59.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi131-83167-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:52:59.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-82449-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:59.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-82060-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-05T07:52:59.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:59.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-82128-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-82128-10"}]': finished 2024-02-05T07:52:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: pgmap v88: 916 pgs: 153 creating+peering, 506 active+clean, 257 unknown; 145 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-02-05T07:52:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-82218-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-82218-10"}]': finished 2024-02-05T07:52:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi131-83124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]': finished 2024-02-05T07:52:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi131-83167-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-4", "tierpool": "test-rados-api-smithi131-82290-4-cache"}]': finished 2024-02-05T07:52:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-82399-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1090436836' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi131-83167-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:52:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4131594549' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi131-83124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:52:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-82399-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-82449-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1051715549' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-82060-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-05T07:52:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:59.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: pgmap v88: 916 pgs: 153 creating+peering, 506 active+clean, 257 unknown; 145 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-02-05T07:52:59.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-82218-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-82218-10"}]': finished 2024-02-05T07:52:59.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi131-83124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:59.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-82531-1"}]': finished 2024-02-05T07:52:59.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi131-83167-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:59.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-4", "tierpool": "test-rados-api-smithi131-82290-4-cache"}]': finished 2024-02-05T07:52:59.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-82399-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:59.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:52:59.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-82060-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:52:59.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1096713826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:59.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1090436836' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi131-83167-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:52:59.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4131594549' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi131-83124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:52:59.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-82399-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:59.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-82449-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:59.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1051715549' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-82060-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-05T07:52:59.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:59.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: osdmap e86: 8 total, 8 up, 8 in 2024-02-05T07:52:59.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-05T07:52:59.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi131-83124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:52:59.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:59.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-4", "overlaypool": "test-rados-api-smithi131-82290-4-cache"}]: dispatch 2024-02-05T07:52:59.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-82399-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:59.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi131-83167-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:52:59.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-82449-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:59.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-82060-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-05T07:52:59.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:59.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-82128-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-82128-10"}]': finished 2024-02-05T07:52:59.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: osdmap e86: 8 total, 8 up, 8 in 2024-02-05T07:52:59.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-05T07:52:59.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi131-83124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:52:59.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:59.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-4", "overlaypool": "test-rados-api-smithi131-82290-4-cache"}]: dispatch 2024-02-05T07:52:59.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-82399-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:59.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi131-83167-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:52:59.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-82449-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:59.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-82060-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-05T07:52:59.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:52:59.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-82128-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-82128-10"}]': finished 2024-02-05T07:52:59.882 INFO:tasks.workunit.client.0.smithi131.stdout:000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-05T07:52:59.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.413558+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi131-83124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:52:59.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.414575+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]: dispatch 2024-02-05T07:52:59.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.416419+0000 mon.a [INF] from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-4", "overlaypool": "test-rados-api-smithi131-82290-4-cache"}]: dispatch 2024-02-05T07:52:59.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.419548+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-82399-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:52:59.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.420558+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi131-83167-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:52:59.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.421094+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-82449-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:52:59.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:56.422660+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-82060-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-05T07:52:59.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:57.160840+0000 mon.a [INF] from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-82128-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-82128-10"}]': finished 2024-02-05T07:52:59.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:57.160930+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]': finished 2024-02-05T07:52:59.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:57.161021+0000 mon.a [INF] from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-4", "overlaypool": "test-rados-api-smithi131-82290-4-cache"}]': finished 2024-02-05T07:52:59.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:57.161096+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-82060-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-05T07:52:59.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:58.154923+0000 mon.c [INF] from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:52:59.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:58.155326+0000 mon.c [INF] from='client.? 172.21.15.131:0/1025281770' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:52:59.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:58.155803+0000 mon.c [INF] from='client.? 172.21.15.131:0/3130449189' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: 12.9 deep-scrub starts 2024-02-05T07:53:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: 12.9 deep-scrub ok 2024-02-05T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: 12.7 deep-scrub starts 2024-02-05T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: 12.7 deep-scrub ok 2024-02-05T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]': finished 2024-02-05T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-4", "overlaypool": "test-rados-api-smithi131-82290-4-cache"}]': finished 2024-02-05T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-82060-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-05T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: pgmap v91: 820 pgs: 44 creating+peering, 472 active+clean, 304 unknown; 145 MiB data, 716 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: 12.4 deep-scrub starts 2024-02-05T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: 12.4 deep-scrub ok 2024-02-05T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1025281770' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3130449189' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/784167816' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-05T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: threexx 2024-02-05T07:53:00.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: osdmap e87: 8 total, 8 up, 8 in 2024-02-05T07:53:00.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-4-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:53:00.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:53:00.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi131-83124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi131-83124-2"}]': finished 2024-02-05T07:53:00.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-82399-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-82399-7"}]': finished 2024-02-05T07:53:00.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi131-83167-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi131-83167-2"}]': finished 2024-02-05T07:53:00.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-82449-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-82449-7"}]': finished 2024-02-05T07:53:00.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2028690928' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-82038-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: 12.1 deep-scrub starts 2024-02-05T07:53:00.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: 12.1 deep-scrub ok 2024-02-05T07:53:00.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:00.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: osdmap e88: 8 total, 8 up, 8 in 2024-02-05T07:53:00.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:00.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:53:00.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-05T07:53:00.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:00.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:52:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-82038-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: 12.9 deep-scrub starts 2024-02-05T07:53:00.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: 12.9 deep-scrub ok 2024-02-05T07:53:00.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: 12.7 deep-scrub starts 2024-02-05T07:53:00.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: 12.7 deep-scrub ok 2024-02-05T07:53:00.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]': finished 2024-02-05T07:53:00.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-4", "overlaypool": "test-rados-api-smithi131-82290-4-cache"}]': finished 2024-02-05T07:53:00.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-82060-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-05T07:53:00.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: pgmap v91: 820 pgs: 44 creating+peering, 472 active+clean, 304 unknown; 145 MiB data, 716 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:53:00.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: 12.4 deep-scrub starts 2024-02-05T07:53:00.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: 12.4 deep-scrub ok 2024-02-05T07:53:00.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:00.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:53:00.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1025281770' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3130449189' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:00.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/784167816' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-05T07:53:00.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: threexx 2024-02-05T07:53:00.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: osdmap e87: 8 total, 8 up, 8 in 2024-02-05T07:53:00.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-4-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:53:00.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:53:00.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi131-83124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi131-83124-2"}]': finished 2024-02-05T07:53:00.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-82399-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-82399-7"}]': finished 2024-02-05T07:53:00.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi131-83167-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi131-83167-2"}]': finished 2024-02-05T07:53:00.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: 12.9 deep-scrub starts 2024-02-05T07:53:00.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: 12.9 deep-scrub ok 2024-02-05T07:53:00.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: 12.7 deep-scrub starts 2024-02-05T07:53:00.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: 12.7 deep-scrub ok 2024-02-05T07:53:00.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-82531-1"}]': finished 2024-02-05T07:53:00.273 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-4", "overlaypool": "test-rados-api-smithi131-82290-4-cache"}]': finished 2024-02-05T07:53:00.273 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-82060-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-05T07:53:00.273 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: pgmap v91: 820 pgs: 44 creating+peering, 472 active+clean, 304 unknown; 145 MiB data, 716 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:53:00.273 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: 12.4 deep-scrub starts 2024-02-05T07:53:00.273 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: 12.4 deep-scrub ok 2024-02-05T07:53:00.273 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:00.274 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:53:00.274 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1025281770' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.274 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3130449189' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.274 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:00.274 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/784167816' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-05T07:53:00.274 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: threexx 2024-02-05T07:53:00.274 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: osdmap e87: 8 total, 8 up, 8 in 2024-02-05T07:53:00.274 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-4-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:53:00.275 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:53:00.275 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi131-83124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi131-83124-2"}]': finished 2024-02-05T07:53:00.275 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-82399-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-82399-7"}]': finished 2024-02-05T07:53:00.275 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi131-83167-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi131-83167-2"}]': finished 2024-02-05T07:53:00.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-82449-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-82449-7"}]': finished 2024-02-05T07:53:00.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2028690928' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-82038-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: 12.1 deep-scrub starts 2024-02-05T07:53:00.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: 12.1 deep-scrub ok 2024-02-05T07:53:00.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:00.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: osdmap e88: 8 total, 8 up, 8 in 2024-02-05T07:53:00.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:00.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:53:00.277 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.277 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.277 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-05T07:53:00.277 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:00.277 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-82038-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.277 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-82449-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-82449-7"}]': finished 2024-02-05T07:53:00.278 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2028690928' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-82038-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.278 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: 12.1 deep-scrub starts 2024-02-05T07:53:00.278 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: 12.1 deep-scrub ok 2024-02-05T07:53:00.278 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:00.278 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: osdmap e88: 8 total, 8 up, 8 in 2024-02-05T07:53:00.279 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:00.279 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:53:00.279 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.279 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:00.279 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-05T07:53:00.279 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:00.279 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:52:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-82038-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:01.004 INFO:tasks.workunit.client.0.smithi131.stdout: api_ ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-02-05T07:53:01.004 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (4 ms) 2024-02-05T07:53:01.004 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-02-05T07:53:01.005 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (6 ms) 2024-02-05T07:53:01.005 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-02-05T07:53:01.005 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (4 ms) 2024-02-05T07:53:01.005 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-02-05T07:53:01.005 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (7 ms) 2024-02-05T07:53:01.005 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-02-05T07:53:01.005 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (3 ms) 2024-02-05T07:53:01.005 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-02-05T07:53:01.005 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (3 ms) 2024-02-05T07:53:01.005 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-02-05T07:53:01.005 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (3 ms) 2024-02-05T07:53:01.005 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-02-05T07:53:01.005 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (8 ms) 2024-02-05T07:53:01.005 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-02-05T07:53:01.005 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (3 ms) 2024-02-05T07:53:01.006 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-02-05T07:53:01.006 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (4 ms) 2024-02-05T07:53:01.006 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-02-05T07:53:01.006 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (2 ms) 2024-02-05T07:53:01.006 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-02-05T07:53:01.006 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (5 ms) 2024-02-05T07:53:01.006 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (2269 ms total) 2024-02-05T07:53:01.006 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: 2024-02-05T07:53:01.006 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [----------] Global test environment tear-down 2024-02-05T07:53:01.006 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (22281 ms total) 2024-02-05T07:53:01.006 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-02-05T07:53:01.011 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: Running main() from gmock_main.cc 2024-02-05T07:53:01.011 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-02-05T07:53:01.011 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [----------] Global test environment set-up. 2024-02-05T07:53:01.012 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-02-05T07:53:01.012 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-02-05T07:53:01.012 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (1282 ms) 2024-02-05T07:53:01.012 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-02-05T07:53:01.012 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-02-05T07:53:01.012 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-02-05T07:53:01.012 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: no timeout :/ 2024-02-05T07:53:01.012 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: no timeout :/ 2024-02-05T07:53:01.012 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: no timeout :/ 2024-02-05T07:53:01.012 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: no timeout :/ 2024-02-05T07:53:01.012 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: no timeout :/ 2024-02-05T07:53:01.012 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (102 ms) 2024-02-05T07:53:01.012 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-02-05T07:53:01.012 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (11 ms) 2024-02-05T07:53:01.012 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-02-05T07:53:01.013 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.Checksum (4 ms) 2024-02-05T07:53:01.013 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-02-05T07:53:01.013 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (6 ms) 2024-02-05T07:53:01.013 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-02-05T07:53:01.013 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (6 ms) 2024-02-05T07:53:01.013 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-02-05T07:53:01.013 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (7 ms) 2024-02-05T07:53:01.013 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-02-05T07:53:01.013 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (3 ms) 2024-02-05T07:53:01.013 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-02-05T07:53:01.013 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.TruncTest (5 ms) 2024-02-05T07:53:01.013 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-02-05T07:53:01.013 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (5 ms) 2024-02-05T07:53:01.013 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-02-05T07:53:01.013 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (6 ms) 2024-02-05T07:53:01.014 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-02-05T07:53:01.014 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.RmXattr (10 ms) 2024-02-05T07:53:01.014 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-02-05T07:53:01.014 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.XattrIter (6 ms) 2024-02-05T07:53:01.014 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [----------] 14 tests from LibRadosIo (1453 ms total) 2024-02-05T07:53:01.014 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: 2024-02-05T07:53:01.014 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-02-05T07:53:01.014 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-02-05T07:53:01.014 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1140 ms) 2024-02-05T07:53:01.014 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-02-05T07:53:01.014 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (1061 ms) 2024-02-05T07:53:01.014 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-02-05T07:53:01.014 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (4 ms) 2024-02-05T07:53:01.014 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-02-05T07:53:01.014 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (6 ms) 2024-02-05T07:53:01.015 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-02-05T07:53:01.015 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (8 ms) 2024-02-05T07:53:01.015 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-02-05T07:53:01.015 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (4 ms) 2024-02-05T07:53:01.015 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-02-05T07:53:01.015 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-02-05T07:53:01.015 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-02-05T07:53:01.015 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (4 ms) 2024-02-05T07:53:01.015 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-02-05T07:53:01.015 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (9 ms) 2024-02-05T07:53:01.015 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-02-05T07:53:01.015 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (4 ms) 2024-02-05T07:53:01.015 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [----------] 10 tests from LibRadosIoEC (2244 ms total) 2024-02-05T07:53:01.015 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: 2024-02-05T07:53:01.015 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [----------] Global test environment tear-down 2024-02-05T07:53:01.016 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [==========] 24 tests from 2 test suites ran. (22297 ms total) 2024-02-05T07:53:01.016 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ PASSED ] 24 tests. 2024-02-05T07:53:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: pgmap v93: 916 pgs: 65 creating+peering, 684 active+clean, 167 unknown; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-02-05T07:53:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-05T07:53:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/784167816' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-05T07:53:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: fourxx 2024-02-05T07:53:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-05T07:53:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:53:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-4-cache", "mode": "writeback"}]': finished 2024-02-05T07:53:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22"}]': finished 2024-02-05T07:53:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]': finished 2024-02-05T07:53:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-82038-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: osdmap e89: 8 total, 8 up, 8 in 2024-02-05T07:53:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:53:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:53:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:53:01.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:53:01.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:01.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:01.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-4"}]: dispatch 2024-02-05T07:53:01.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:01.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:01.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:01.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:01.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: pgmap v93: 916 pgs: 65 creating+peering, 684 active+clean, 167 unknown; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-02-05T07:53:01.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-05T07:53:01.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/784167816' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-05T07:53:01.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: fourxx 2024-02-05T07:53:01.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-05T07:53:01.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:53:01.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-4-cache", "mode": "writeback"}]': finished 2024-02-05T07:53:01.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22"}]': finished 2024-02-05T07:53:01.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]': finished 2024-02-05T07:53:01.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:01.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:01.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-82038-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:01.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: osdmap e89: 8 total, 8 up, 8 in 2024-02-05T07:53:01.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:53:01.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:53:01.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:01.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:01.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:01.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:53:01.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:53:01.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:01.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:01.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-4"}]: dispatch 2024-02-05T07:53:01.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:01.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:01.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:01.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:01.328 INFO:tasks.workunit.client.0.smithi131.stdout: list: Running main() from gmock_main.cc 2024-02-05T07:53:01.328 INFO:tasks.workunit.client.0.smithi131.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-02-05T07:53:01.328 INFO:tasks.workunit.client.0.smithi131.stdout: list: [----------] Global test environment set-up. 2024-02-05T07:53:01.328 INFO:tasks.workunit.client.0.smithi131.stdout: list: [----------] 3 tests from NeoradosList 2024-02-05T07:53:01.328 INFO:tasks.workunit.client.0.smithi131.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-02-05T07:53:01.328 INFO:tasks.workunit.client.0.smithi131.stdout: list: [ OK ] NeoradosList.ListObjects (10761 ms) 2024-02-05T07:53:01.328 INFO:tasks.workunit.client.0.smithi131.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-02-05T07:53:01.328 INFO:tasks.workunit.client.0.smithi131.stdout: list: [ OK ] NeoradosList.ListObjectsNS (6544 ms) 2024-02-05T07:53:01.328 INFO:tasks.workunit.client.0.smithi131.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-02-05T07:53:01.328 INFO:tasks.workunit.client.0.smithi131.stdout: list: [ OK ] NeoradosList.ListObjectsMany (4924 ms) 2024-02-05T07:53:01.329 INFO:tasks.workunit.client.0.smithi131.stdout: list: [----------] 3 tests from NeoradosList (22229 ms total) 2024-02-05T07:53:01.329 INFO:tasks.workunit.client.0.smithi131.stdout: list: 2024-02-05T07:53:01.329 INFO:tasks.workunit.client.0.smithi131.stdout: list: [----------] Global test environment tear-down 2024-02-05T07:53:01.329 INFO:tasks.workunit.client.0.smithi131.stdout: list: [==========] 3 tests from 1 test suite ran. (22229 ms total) 2024-02-05T07:53:01.329 INFO:tasks.workunit.client.0.smithi131.stdout: list: [ PASSED ] 3 tests. 2024-02-05T07:53:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: pgmap v93: 916 pgs: 65 creating+peering, 684 active+clean, 167 unknown; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-02-05T07:53:01.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-05T07:53:01.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/784167816' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-05T07:53:01.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: fourxx 2024-02-05T07:53:01.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-05T07:53:01.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:53:01.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-4-cache", "mode": "writeback"}]': finished 2024-02-05T07:53:01.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22"}]': finished 2024-02-05T07:53:01.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]': finished 2024-02-05T07:53:01.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:01.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:01.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-82038-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:01.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: osdmap e89: 8 total, 8 up, 8 in 2024-02-05T07:53:01.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3713608241' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:53:01.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:53:01.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:01.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:01.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:01.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:53:01.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:53:01.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:01.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:01.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-4"}]: dispatch 2024-02-05T07:53:01.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:01.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:01.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:01.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:02.083 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-02-05T07:53:02.083 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-02-05T07:53:02.083 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-02-05T07:53:02.083 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-02-05T07:53:02.083 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: seed 82218 2024-02-05T07:53:02.083 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-02-05T07:53:02.083 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (249 ms) 2024-02-05T07:53:02.083 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-02-05T07:53:02.083 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (12 ms) 2024-02-05T07:53:02.083 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-02-05T07:53:02.083 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1109 ms) 2024-02-05T07:53:02.084 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-02-05T07:53:02.084 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1045 ms) 2024-02-05T07:53:02.084 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-02-05T07:53:02.084 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (5 ms) 2024-02-05T07:53:02.084 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-02-05T07:53:02.084 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (4 ms) 2024-02-05T07:53:02.084 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-02-05T07:53:02.084 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (8 ms) 2024-02-05T07:53:02.084 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-02-05T07:53:02.084 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (25 ms) 2024-02-05T07:53:02.084 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2457 ms total) 2024-02-05T07:53:02.084 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: 2024-02-05T07:53:02.084 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-02-05T07:53:02.084 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-02-05T07:53:02.084 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (830 ms) 2024-02-05T07:53:02.085 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-02-05T07:53:02.085 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (4 ms) 2024-02-05T07:53:02.085 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-02-05T07:53:02.085 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1004 ms) 2024-02-05T07:53:02.085 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-02-05T07:53:02.085 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1005 ms) 2024-02-05T07:53:02.085 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-02-05T07:53:02.085 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (4 ms) 2024-02-05T07:53:02.085 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-02-05T07:53:02.085 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (4 ms) 2024-02-05T07:53:02.085 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-02-05T07:53:02.085 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (4 ms) 2024-02-05T07:53:02.085 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-02-05T07:53:02.085 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (4 ms) 2024-02-05T07:53:02.085 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (2859 ms total) 2024-02-05T07:53:02.086 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: 2024-02-05T07:53:02.086 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-02-05T07:53:02.086 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (23317 ms total) 2024-02-05T07:53:02.086 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-02-05T07:53:02.647 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-05T07:53:02.647 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-82092-22"}]': finished 2024-02-05T07:53:02.647 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]': finished 2024-02-05T07:53:02.647 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10"}]': finished 2024-02-05T07:53:02.647 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:02.647 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-4"}]': finished 2024-02-05T07:53:02.647 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:02.647 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: osdmap e90: 8 total, 8 up, 8 in 2024-02-05T07:53:02.647 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:53:02.647 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:53:02.648 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:53:02.648 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-4", "tierpool": "test-rados-api-smithi131-82290-4-cache"}]: dispatch 2024-02-05T07:53:02.648 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4131594549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:53:02.648 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:53:02.648 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:53:02.648 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:53:02.648 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:02.648 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:02.648 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1090436836' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:53:02.648 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:53:02.648 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82531-6", "pg_num": 4}]: dispatch 2024-02-05T07:53:02.648 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:02.648 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: Health check update: 19 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:02.648 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: pool 'PoolQuotaPP_smithi131-82060-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-05T07:53:02.649 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T07:53:02.649 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:53:02.649 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-05T07:53:02.649 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-82092-22"}]': finished 2024-02-05T07:53:02.649 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]': finished 2024-02-05T07:53:02.649 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10"}]': finished 2024-02-05T07:53:02.649 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:02.649 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-4"}]': finished 2024-02-05T07:53:02.649 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:02.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: osdmap e90: 8 total, 8 up, 8 in 2024-02-05T07:53:02.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:53:02.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:53:02.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:53:02.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-4", "tierpool": "test-rados-api-smithi131-82290-4-cache"}]: dispatch 2024-02-05T07:53:02.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4131594549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:53:02.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:53:02.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:53:02.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:53:02.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:02.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:02.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1090436836' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:53:02.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:53:02.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82531-6", "pg_num": 4}]: dispatch 2024-02-05T07:53:02.650 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:02.651 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: Health check update: 19 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:02.651 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: pool 'PoolQuotaPP_smithi131-82060-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-05T07:53:02.651 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T07:53:02.651 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:53:02.652 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-05T07:53:02.652 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-82092-22"}]': finished 2024-02-05T07:53:02.652 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-82071-16"}]': finished 2024-02-05T07:53:02.652 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-82218-10"}]': finished 2024-02-05T07:53:02.652 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:02.652 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-4"}]': finished 2024-02-05T07:53:02.652 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:02.653 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: osdmap e90: 8 total, 8 up, 8 in 2024-02-05T07:53:02.653 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:53:02.653 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-82218-10"}]: dispatch 2024-02-05T07:53:02.653 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:53:02.653 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-4", "tierpool": "test-rados-api-smithi131-82290-4-cache"}]: dispatch 2024-02-05T07:53:02.653 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4131594549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:53:02.653 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:53:02.653 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:53:02.653 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:53:02.653 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:02.653 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:02.653 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1090436836' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:53:02.653 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:53:02.653 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82531-6", "pg_num": 4}]: dispatch 2024-02-05T07:53:02.653 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:02.654 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: Health check update: 19 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:02.654 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: pool 'PoolQuotaPP_smithi131-82060-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-05T07:53:02.654 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T07:53:02.654 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:02 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:53:02.980 INFO:tasks.workunit.client.0.smithi131.stdout:cmd: got: 2024-02-05T07:52:58.413007+0000 mon.c [INF] from='client.? 172.21.15.131:0/784167816' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-05T07:53:02.980 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:58.413163+0000 client.admin [INF] threexx 2024-02-05T07:53:02.980 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:58.874633+0000 mon.a [INF] from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-4-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:53:02.980 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:58.876839+0000 mon.a [INF] from='client.? 172.21.15.131:0/88550618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-82092-22"}]: dispatch 2024-02-05T07:53:02.980 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:58.889177+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi131-83124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi131-83124-2"}]': finished 2024-02-05T07:53:02.980 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:58.889248+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-82399-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-82399-7"}]': finished 2024-02-05T07:53:02.980 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:58.889307+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi131-83167-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi131-83167-2"}]': finished 2024-02-05T07:53:02.980 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:58.889368+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-82449-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-82449-7"}]': finished 2024-02-05T07:53:02.980 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:58.896290+0000 mon.b [INF] from='client.? 172.21.15.131:0/2028690928' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-82038-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:02.981 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:59.149071+0000 mon.a [INF] from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:02.981 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:59.149971+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-82071-16"}]: dispatch 2024-02-05T07:53:02.981 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:59.151391+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:02.981 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:59.153394+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:02.981 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:59.154547+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-05T07:53:02.981 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-02-05T07:52:59.160646+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-82038-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:02.981 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (10961 ms) 2024-02-05T07:53:02.981 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (24093 ms total) 2024-02-05T07:53:02.981 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: 2024-02-05T07:53:02.981 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [----------] Global test environment tear-down 2024-02-05T07:53:02.981 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (24093 ms total) 2024-02-05T07:53:02.981 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ PASSED ] 4 tests. 2024-02-05T07:53:03.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:53:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:53:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:53:03.384 INFO:tasks.workunit.client.0.smithi131.stdout: ec_io: Running main() from gmock_main.cc 2024-02-05T07:53:03.385 INFO:tasks.workunit.client.0.smithi131.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-02-05T07:53:03.385 INFO:tasks.workunit.client.0.smithi131.stdout: ec_io: [----------] Global test environment set-up. 2024-02-05T07:53:03.385 INFO:tasks.workunit.client.0.smithi131.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-02-05T07:53:03.385 INFO:tasks.workunit.client.0.smithi131.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-02-05T07:53:03.385 INFO:tasks.workunit.client.0.smithi131.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (14945 ms) 2024-02-05T07:53:03.385 INFO:tasks.workunit.client.0.smithi131.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-02-05T07:53:03.385 INFO:tasks.workunit.client.0.smithi131.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (9353 ms) 2024-02-05T07:53:03.385 INFO:tasks.workunit.client.0.smithi131.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (24298 ms total) 2024-02-05T07:53:03.385 INFO:tasks.workunit.client.0.smithi131.stdout: ec_io: 2024-02-05T07:53:03.385 INFO:tasks.workunit.client.0.smithi131.stdout: ec_io: [----------] Global test environment tear-down 2024-02-05T07:53:03.385 INFO:tasks.workunit.client.0.smithi131.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (24298 ms total) 2024-02-05T07:53:03.385 INFO:tasks.workunit.client.0.smithi131.stdout: ec_io: [ PASSED ] 2 tests. 2024-02-05T07:53:03.398 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: Running main() from gmock_main.cc 2024-02-05T07:53:03.398 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-02-05T07:53:03.398 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [----------] Global test environment set-up. 2024-02-05T07:53:03.398 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-02-05T07:53:03.398 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-02-05T07:53:03.398 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStat.Stat (256 ms) 2024-02-05T07:53:03.398 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-02-05T07:53:03.398 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (7 ms) 2024-02-05T07:53:03.398 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-02-05T07:53:03.398 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStat.StatNS (20 ms) 2024-02-05T07:53:03.399 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-02-05T07:53:03.399 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (1 ms) 2024-02-05T07:53:03.399 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-02-05T07:53:03.399 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (3 ms) 2024-02-05T07:53:03.399 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [----------] 5 tests from LibRadosStat (287 ms total) 2024-02-05T07:53:03.399 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: 2024-02-05T07:53:03.399 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-02-05T07:53:03.399 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-02-05T07:53:03.399 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1154 ms) 2024-02-05T07:53:03.399 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-02-05T07:53:03.399 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (11 ms) 2024-02-05T07:53:03.399 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-02-05T07:53:03.399 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (1 ms) 2024-02-05T07:53:03.399 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-02-05T07:53:03.400 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (2 ms) 2024-02-05T07:53:03.400 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1168 ms total) 2024-02-05T07:53:03.400 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: 2024-02-05T07:53:03.400 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [----------] Global test environment tear-down 2024-02-05T07:53:03.400 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (24553 ms total) 2024-02-05T07:53:03.400 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ PASSED ] 9 tests. 2024-02-05T07:53:03.400 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-02-05T07:53:03.400 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-02-05T07:53:03.400 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-02-05T07:53:03.400 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-02-05T07:53:03.400 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: seed 82449 2024-02-05T07:53:03.400 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-02-05T07:53:03.400 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (248 ms) 2024-02-05T07:53:03.400 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-02-05T07:53:03.401 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (9 ms) 2024-02-05T07:53:03.401 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-02-05T07:53:03.401 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (1 ms) 2024-02-05T07:53:03.401 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-02-05T07:53:03.401 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (3 ms) 2024-02-05T07:53:03.401 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-02-05T07:53:03.401 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (16 ms) 2024-02-05T07:53:03.401 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (277 ms total) 2024-02-05T07:53:03.401 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: 2024-02-05T07:53:03.401 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-02-05T07:53:03.401 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-02-05T07:53:03.401 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1154 ms) 2024-02-05T07:53:03.401 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-02-05T07:53:03.401 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (1 ms) 2024-02-05T07:53:03.401 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-02-05T07:53:03.402 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (5 ms) 2024-02-05T07:53:03.402 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-02-05T07:53:03.402 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (9 ms) 2024-02-05T07:53:03.402 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1169 ms total) 2024-02-05T07:53:03.402 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: 2024-02-05T07:53:03.402 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-02-05T07:53:03.402 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (24544 ms total) 2024-02-05T07:53:03.402 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-02-05T07:53:03.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: pgmap v96: 716 pgs: 7 creating+peering, 516 active+clean, 193 unknown; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:53:03.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-82218-10"}]': finished 2024-02-05T07:53:03.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-4", "tierpool": "test-rados-api-smithi131-82290-4-cache"}]': finished 2024-02-05T07:53:03.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]': finished 2024-02-05T07:53:03.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]': finished 2024-02-05T07:53:03.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi131-83124-2"}]': finished 2024-02-05T07:53:03.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi131-83167-2"}]': finished 2024-02-05T07:53:03.714 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:53:03.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4131594549' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:53:03.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:53:03.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1090436836' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:53:03.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:53:03.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: osdmap e91: 8 total, 8 up, 8 in 2024-02-05T07:53:03.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82531-6", "pg_num": 4}]: dispatch 2024-02-05T07:53:03.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:03.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:53:03.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]': finished 2024-02-05T07:53:03.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82531-6", "pg_num": 4}]': finished 2024-02-05T07:53:03.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:03.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2682595705' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-82038-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:03.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: osdmap e92: 8 total, 8 up, 8 in 2024-02-05T07:53:03.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:53:03.715 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:53:03.716 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:53:03.716 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:03.716 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-82038-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:03.716 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: pgmap v96: 716 pgs: 7 creating+peering, 516 active+clean, 193 unknown; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:53:03.716 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-82218-10"}]': finished 2024-02-05T07:53:03.716 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-4", "tierpool": "test-rados-api-smithi131-82290-4-cache"}]': finished 2024-02-05T07:53:03.716 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]': finished 2024-02-05T07:53:03.716 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]': finished 2024-02-05T07:53:03.716 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi131-83124-2"}]': finished 2024-02-05T07:53:03.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi131-83167-2"}]': finished 2024-02-05T07:53:03.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:53:03.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4131594549' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:53:03.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:53:03.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1090436836' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:53:03.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:53:03.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: osdmap e91: 8 total, 8 up, 8 in 2024-02-05T07:53:03.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82531-6", "pg_num": 4}]: dispatch 2024-02-05T07:53:03.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:03.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:53:03.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]': finished 2024-02-05T07:53:03.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82531-6", "pg_num": 4}]': finished 2024-02-05T07:53:03.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:03.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2682595705' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-82038-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:03.718 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: osdmap e92: 8 total, 8 up, 8 in 2024-02-05T07:53:03.718 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:53:03.718 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:53:03.718 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:53:03.718 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:03.718 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-82038-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:03.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: pgmap v96: 716 pgs: 7 creating+peering, 516 active+clean, 193 unknown; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:53:03.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/593133211' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-82218-10"}]': finished 2024-02-05T07:53:03.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1050208429' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-4", "tierpool": "test-rados-api-smithi131-82290-4-cache"}]': finished 2024-02-05T07:53:03.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-82399-7"}]': finished 2024-02-05T07:53:03.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-82449-7"}]': finished 2024-02-05T07:53:03.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi131-83124-2"}]': finished 2024-02-05T07:53:03.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi131-83167-2"}]': finished 2024-02-05T07:53:03.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2718893104' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:53:03.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4131594549' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:53:03.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1007450002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:53:03.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1090436836' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:53:03.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:53:03.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: osdmap e91: 8 total, 8 up, 8 in 2024-02-05T07:53:03.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82531-6", "pg_num": 4}]: dispatch 2024-02-05T07:53:03.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:03.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi131-83167-2"}]: dispatch 2024-02-05T07:53:03.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-82517-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]': finished 2024-02-05T07:53:03.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82531-6", "pg_num": 4}]': finished 2024-02-05T07:53:03.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:03.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2682595705' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-82038-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:03.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: osdmap e92: 8 total, 8 up, 8 in 2024-02-05T07:53:03.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]: dispatch 2024-02-05T07:53:03.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi131-83124-2"}]: dispatch 2024-02-05T07:53:03.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]: dispatch 2024-02-05T07:53:03.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:03.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-82038-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:04.665 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:04.665 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: pgmap v99: 728 pgs: 108 creating+peering, 620 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-02-05T07:53:04.665 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi131-83167-2"}]': finished 2024-02-05T07:53:04.665 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]': finished 2024-02-05T07:53:04.665 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi131-83124-2"}]': finished 2024-02-05T07:53:04.665 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]': finished 2024-02-05T07:53:04.665 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:53:04.665 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-82038-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:04.666 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: osdmap e93: 8 total, 8 up, 8 in 2024-02-05T07:53:04.666 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2982926900' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:04.666 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "overlaypool": "test-rados-api-smithi131-82531-6"}]: dispatch 2024-02-05T07:53:04.666 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1225171578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:04.666 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "overlaypool": "test-rados-api-smithi131-82531-6"}]: dispatch 2024-02-05T07:53:04.666 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:53:04.666 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:04.666 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:04.666 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3149099367' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi131-83167-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:04.666 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi131-83167-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:04.666 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:04.666 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:04.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: pgmap v99: 728 pgs: 108 creating+peering, 620 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-02-05T07:53:04.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi131-83167-2"}]': finished 2024-02-05T07:53:04.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]': finished 2024-02-05T07:53:04.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi131-83124-2"}]': finished 2024-02-05T07:53:04.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]': finished 2024-02-05T07:53:04.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:53:04.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-82038-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:04.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: osdmap e93: 8 total, 8 up, 8 in 2024-02-05T07:53:04.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2982926900' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:04.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "overlaypool": "test-rados-api-smithi131-82531-6"}]: dispatch 2024-02-05T07:53:04.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1225171578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:04.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "overlaypool": "test-rados-api-smithi131-82531-6"}]: dispatch 2024-02-05T07:53:04.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:53:04.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:04.667 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:04.668 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3149099367' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi131-83167-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:04.668 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi131-83167-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:04.668 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:04.700 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:04.701 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: pgmap v99: 728 pgs: 108 creating+peering, 620 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-02-05T07:53:04.701 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi131-83167-2"}]': finished 2024-02-05T07:53:04.701 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-82449-7"}]': finished 2024-02-05T07:53:04.701 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi131-83124-2"}]': finished 2024-02-05T07:53:04.701 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-82399-7"}]': finished 2024-02-05T07:53:04.701 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:53:04.701 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-82038-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:04.701 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: osdmap e93: 8 total, 8 up, 8 in 2024-02-05T07:53:04.701 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2982926900' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:04.701 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "overlaypool": "test-rados-api-smithi131-82531-6"}]: dispatch 2024-02-05T07:53:04.701 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1225171578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:04.701 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "overlaypool": "test-rados-api-smithi131-82531-6"}]: dispatch 2024-02-05T07:53:04.701 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:53:04.701 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:04.701 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:04.702 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3149099367' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi131-83167-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:04.702 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi131-83167-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:04.702 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:05.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: Running main() from gmock_main.cc 2024-02-05T07:53:05.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-02-05T07:53:05.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [----------] Global test environment set-up. 2024-02-05T07:53:05.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-02-05T07:53:05.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-02-05T07:53:05.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (1280 ms) 2024-02-05T07:53:05.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-02-05T07:53:05.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLock.LockShared (6 ms) 2024-02-05T07:53:05.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-02-05T07:53:05.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1074 ms) 2024-02-05T07:53:05.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-02-05T07:53:05.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1052 ms) 2024-02-05T07:53:05.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-02-05T07:53:05.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (4 ms) 2024-02-05T07:53:05.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-02-05T07:53:05.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLock.Unlock (14 ms) 2024-02-05T07:53:05.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-02-05T07:53:05.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (43 ms) 2024-02-05T07:53:05.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-02-05T07:53:05.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-02-05T07:53:05.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [----------] 8 tests from LibRadosLock (3476 ms total) 2024-02-05T07:53:05.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: 2024-02-05T07:53:05.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-02-05T07:53:05.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-02-05T07:53:05.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1158 ms) 2024-02-05T07:53:05.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-02-05T07:53:05.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (19 ms) 2024-02-05T07:53:05.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-02-05T07:53:05.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1048 ms) 2024-02-05T07:53:05.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-02-05T07:53:05.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1004 ms) 2024-02-05T07:53:05.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-02-05T07:53:05.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (5 ms) 2024-02-05T07:53:05.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-02-05T07:53:05.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (4 ms) 2024-02-05T07:53:05.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-02-05T07:53:05.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (6 ms) 2024-02-05T07:53:05.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-02-05T07:53:05.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (4 ms) 2024-02-05T07:53:05.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3248 ms total) 2024-02-05T07:53:05.724 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: 2024-02-05T07:53:05.724 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [----------] Global test environment tear-down 2024-02-05T07:53:05.724 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (26986 ms total) 2024-02-05T07:53:05.724 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ PASSED ] 16 tests. 2024-02-05T07:53:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "overlaypool": "test-rados-api-smithi131-82531-6"}]': finished 2024-02-05T07:53:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-82128-10"}]': finished 2024-02-05T07:53:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi131-83167-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: osdmap e94: 8 total, 8 up, 8 in 2024-02-05T07:53:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:53:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82531-6", "mode": "writeback"}]: dispatch 2024-02-05T07:53:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3149099367' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi131-83167-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:06.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2965319342' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-82377-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:06.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82531-6", "mode": "writeback"}]: dispatch 2024-02-05T07:53:06.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi131-83167-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:06.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:06.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-82377-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:06.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:06.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "overlaypool": "test-rados-api-smithi131-82531-6"}]': finished 2024-02-05T07:53:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-82128-10"}]': finished 2024-02-05T07:53:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi131-83167-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:06.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: osdmap e94: 8 total, 8 up, 8 in 2024-02-05T07:53:06.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:53:06.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82531-6", "mode": "writeback"}]: dispatch 2024-02-05T07:53:06.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3149099367' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi131-83167-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:06.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:06.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2965319342' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-82377-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:06.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82531-6", "mode": "writeback"}]: dispatch 2024-02-05T07:53:06.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi131-83167-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:06.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:06.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-82377-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:06.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:06.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:05 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:06.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "overlaypool": "test-rados-api-smithi131-82531-6"}]': finished 2024-02-05T07:53:06.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-82128-10"}]': finished 2024-02-05T07:53:06.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:06.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:06.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi131-83167-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:06.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: osdmap e94: 8 total, 8 up, 8 in 2024-02-05T07:53:06.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-82128-10"}]: dispatch 2024-02-05T07:53:06.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82531-6", "mode": "writeback"}]: dispatch 2024-02-05T07:53:06.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3149099367' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi131-83167-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:06.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:06.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2965319342' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-82377-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:06.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82531-6", "mode": "writeback"}]: dispatch 2024-02-05T07:53:06.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi131-83167-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:06.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:06.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-82377-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:06.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:06.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:05 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:06.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-02-05T07:53:06.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-02-05T07:53:06.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-02-05T07:53:06.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-02-05T07:53:06.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-02-05T07:53:06.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify_test_cb 2024-02-05T07:53:06.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (1276 ms) 2024-02-05T07:53:06.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-02-05T07:53:06.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94607383000512 err -107 2024-02-05T07:53:06.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-05T07:53:06.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (17 ms) 2024-02-05T07:53:06.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-02-05T07:53:06.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94607383011856 err -107 2024-02-05T07:53:06.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-05T07:53:06.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (14 ms) 2024-02-05T07:53:06.504 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-02-05T07:53:06.504 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_cb from 24622 notify_id 339302416386 cookie 94607383014336 2024-02-05T07:53:06.504 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (11 ms) 2024-02-05T07:53:06.504 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-02-05T07:53:06.504 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_cb from 24622 notify_id 339302416384 cookie 94607383031520 2024-02-05T07:53:06.504 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (8 ms) 2024-02-05T07:53:06.504 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-02-05T07:53:06.504 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_cb from 24622 notify_id 339302416384 cookie 94607383045952 2024-02-05T07:53:06.504 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (10 ms) 2024-02-05T07:53:06.504 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-02-05T07:53:06.504 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_cb from 24622 notify_id 339302416385 cookie 94607383045952 2024-02-05T07:53:06.504 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_cb from 24622 notify_id 339302416385 cookie 94607383063760 2024-02-05T07:53:06.504 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (39 ms) 2024-02-05T07:53:06.504 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-02-05T07:53:06.505 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_cb from 24622 notify_id 339302416385 cookie 94607383045952 2024-02-05T07:53:06.505 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_cb from 24622 notify_id 343597383682 cookie 94607383045952 2024-02-05T07:53:06.505 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3005 ms) 2024-02-05T07:53:06.505 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-02-05T07:53:06.505 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-02-05T07:53:06.505 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94607383045952 err -107 2024-02-05T07:53:06.505 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_cb from 24622 notify_id 369367187458 cookie 94607383045952 2024-02-05T07:53:06.505 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5008 ms) 2024-02-05T07:53:06.505 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-02-05T07:53:06.505 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-02-05T07:53:06.505 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94607383045952 err -107 2024-02-05T07:53:06.505 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1004 ms) 2024-02-05T07:53:06.505 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (10392 ms total) 2024-02-05T07:53:06.505 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: 2024-02-05T07:53:06.505 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-02-05T07:53:06.505 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-02-05T07:53:06.506 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify_test_cb 2024-02-05T07:53:06.506 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1155 ms) 2024-02-05T07:53:06.506 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1156 ms total) 2024-02-05T07:53:06.506 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: 2024-02-05T07:53:06.506 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-02-05T07:53:06.506 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (27627 ms total) 2024-02-05T07:53:06.506 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-02-05T07:53:06.512 INFO:tasks.workunit.client.0.smithi131.stdout: pool: Running main() from gmock_main.cc 2024-02-05T07:53:06.512 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-02-05T07:53:06.512 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [----------] Global test environment set-up. 2024-02-05T07:53:06.512 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-02-05T07:53:06.512 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-02-05T07:53:06.512 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [ OK ] NeoRadosPools.PoolList (9497 ms) 2024-02-05T07:53:06.512 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-02-05T07:53:06.512 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (4400 ms) 2024-02-05T07:53:06.512 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-02-05T07:53:06.512 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (3387 ms) 2024-02-05T07:53:06.512 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-02-05T07:53:06.512 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (4904 ms) 2024-02-05T07:53:06.512 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-02-05T07:53:06.513 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (3080 ms) 2024-02-05T07:53:06.513 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-02-05T07:53:06.513 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (2108 ms) 2024-02-05T07:53:06.513 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [----------] 6 tests from NeoRadosPools (27377 ms total) 2024-02-05T07:53:06.513 INFO:tasks.workunit.client.0.smithi131.stdout: pool: 2024-02-05T07:53:06.513 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [----------] Global test environment tear-down 2024-02-05T07:53:06.513 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [==========] 6 tests from 1 test suite ran. (27377 ms total) 2024-02-05T07:53:06.513 INFO:tasks.workunit.client.0.smithi131.stdout: pool: [ PASSED ] 6 tests. 2024-02-05T07:53:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:06 smithi157 ceph-mon[33345]: pgmap v102: 648 pgs: 160 unknown, 36 creating+peering, 452 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-02-05T07:53:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:06 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:53:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-82128-10"}]': finished 2024-02-05T07:53:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82531-6", "mode": "writeback"}]': finished 2024-02-05T07:53:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]': finished 2024-02-05T07:53:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-82377-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:06 smithi157 ceph-mon[33345]: osdmap e95: 8 total, 8 up, 8 in 2024-02-05T07:53:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2814670123' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-82038-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:06 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-82038-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:06 smithi157 ceph-mon[33345]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi131-83167-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi131-83167-3"}]': finished 2024-02-05T07:53:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[41654]: pgmap v102: 648 pgs: 160 unknown, 36 creating+peering, 452 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-02-05T07:53:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:53:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-82128-10"}]': finished 2024-02-05T07:53:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82531-6", "mode": "writeback"}]': finished 2024-02-05T07:53:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]': finished 2024-02-05T07:53:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-82377-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[41654]: osdmap e95: 8 total, 8 up, 8 in 2024-02-05T07:53:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2814670123' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-82038-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:07.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:07.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:07.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-82038-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:07.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:07.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[41654]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:07.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi131-83167-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi131-83167-3"}]': finished 2024-02-05T07:53:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[29507]: pgmap v102: 648 pgs: 160 unknown, 36 creating+peering, 452 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-02-05T07:53:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:53:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/958870615' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-82128-10"}]': finished 2024-02-05T07:53:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82531-6", "mode": "writeback"}]': finished 2024-02-05T07:53:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-82517-12"}]': finished 2024-02-05T07:53:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-82377-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:07.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3500360314' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:07.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[29507]: osdmap e95: 8 total, 8 up, 8 in 2024-02-05T07:53:07.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2814670123' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-82038-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:07.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-82517-12"}]: dispatch 2024-02-05T07:53:07.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:07.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-82038-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:07.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:07.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[29507]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:07.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi131-83167-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi131-83167-3"}]': finished 2024-02-05T07:53:08.570 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-02-05T07:53:08.570 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-02-05T07:53:08.571 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-02-05T07:53:08.571 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-02-05T07:53:08.571 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-02-05T07:53:08.571 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: notify 2024-02-05T07:53:08.571 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1176 ms) 2024-02-05T07:53:08.571 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-02-05T07:53:08.571 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (4 ms) 2024-02-05T07:53:08.571 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (1180 ms total) 2024-02-05T07:53:08.571 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: 2024-02-05T07:53:08.571 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-02-05T07:53:08.571 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-02-05T07:53:08.571 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: notify 2024-02-05T07:53:08.571 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (110 ms) 2024-02-05T07:53:08.571 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-02-05T07:53:08.572 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: notify 2024-02-05T07:53:08.572 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (4424 ms) 2024-02-05T07:53:08.572 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-02-05T07:53:08.572 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (7 ms) 2024-02-05T07:53:08.572 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-02-05T07:53:08.572 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (3 ms) 2024-02-05T07:53:08.576 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-02-05T07:53:08.576 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94027809438016 notify_id 408021893122 notifier_gid 25238 2024-02-05T07:53:08.576 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (5 ms) 2024-02-05T07:53:08.576 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-02-05T07:53:08.576 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94027809989664 notify_id 408021893120 notifier_gid 25238 2024-02-05T07:53:08.576 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (5 ms) 2024-02-05T07:53:08.576 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-02-05T07:53:08.576 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94027809989664 notify_id 408021893123 notifier_gid 25238 2024-02-05T07:53:08.577 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (4 ms) 2024-02-05T07:53:08.577 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-02-05T07:53:08.577 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94027809989664 notify_id 408021893123 notifier_gid 25238 2024-02-05T07:53:08.577 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (5 ms) 2024-02-05T07:53:08.577 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-02-05T07:53:08.577 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94027810003872 notify_id 408021893123 notifier_gid 25238 2024-02-05T07:53:08.577 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (5 ms) 2024-02-05T07:53:08.578 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-02-05T07:53:08.578 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94027810003872 notify_id 408021893124 notifier_gid 25238 2024-02-05T07:53:08.578 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (4 ms) 2024-02-05T07:53:08.578 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-02-05T07:53:08.578 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: trying... 2024-02-05T07:53:08.578 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94027810016816 notify_id 408021893121 notifier_gid 25238 2024-02-05T07:53:08.578 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: timed out 2024-02-05T07:53:08.579 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: flushing 2024-02-05T07:53:08.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:53:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:53:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-82517-12"}]': finished 2024-02-05T07:53:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-82038-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:08.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:08 smithi157 ceph-mon[33345]: osdmap e96: 8 total, 8 up, 8 in 2024-02-05T07:53:08.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:08.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:08 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:08.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:08 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:08.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:08 smithi157 ceph-mon[33345]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T07:53:08.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:08.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:08 smithi157 ceph-mon[33345]: osdmap e97: 8 total, 8 up, 8 in 2024-02-05T07:53:08.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1322388505' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/301884945' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-82517-12"}]': finished 2024-02-05T07:53:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-82038-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[41654]: osdmap e96: 8 total, 8 up, 8 in 2024-02-05T07:53:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:08.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[41654]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T07:53:08.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:08.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[41654]: osdmap e97: 8 total, 8 up, 8 in 2024-02-05T07:53:08.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1322388505' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/301884945' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-82517-12"}]': finished 2024-02-05T07:53:08.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-82038-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:08.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[29507]: osdmap e96: 8 total, 8 up, 8 in 2024-02-05T07:53:08.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:08.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:08.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:08.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[29507]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T07:53:08.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:08.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[29507]: osdmap e97: 8 total, 8 up, 8 in 2024-02-05T07:53:08.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1322388505' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/301884945' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:08.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_watc 44 creating+peering 2024-02-05T07:53:08.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-02-05T07:53:08.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: io: 2024-02-05T07:53:08.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-05T07:53:08.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-02-05T07:53:08.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-02-05T07:53:08.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ OK ] LibRadosService.StatusFormat (4705 ms) 2024-02-05T07:53:08.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ RUN ] LibRadosService.Status 2024-02-05T07:53:08.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ OK ] LibRadosService.Status (20039 ms) 2024-02-05T07:53:08.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [----------] 4 tests from LibRadosService (29839 ms total) 2024-02-05T07:53:08.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-02-05T07:53:08.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [----------] Global test environment tear-down 2024-02-05T07:53:08.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [==========] 4 tests from 1 test suite ran. (29839 ms total) 2024-02-05T07:53:08.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ PASSED ] 4 tests. 2024-02-05T07:53:09.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:09 smithi157 ceph-mon[33345]: pgmap v105: 656 pgs: 232 unknown, 4 creating+peering, 420 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:09.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:09 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:53:09.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:09 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:53:09.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:09 smithi157 ceph-mon[33345]: pgmap v107: 624 pgs: 232 unknown, 4 creating+peering, 388 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:09.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:09.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:09 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:09.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:09.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:09.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1085585048' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-82038-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:09.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:09 smithi157 ceph-mon[33345]: osdmap e98: 8 total, 8 up, 8 in 2024-02-05T07:53:09.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:09.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-82038-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:09.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:09.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:09 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:09.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[29507]: pgmap v105: 656 pgs: 232 unknown, 4 creating+peering, 420 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:53:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:53:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[29507]: pgmap v107: 624 pgs: 232 unknown, 4 creating+peering, 388 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:09.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:09.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:09.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1085585048' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-82038-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:09.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[29507]: osdmap e98: 8 total, 8 up, 8 in 2024-02-05T07:53:09.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:09.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-82038-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:09.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:09.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:09.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:09.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[41654]: pgmap v105: 656 pgs: 232 unknown, 4 creating+peering, 420 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:09.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:53:09.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:53:09.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[41654]: pgmap v107: 624 pgs: 232 unknown, 4 creating+peering, 388 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:09.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:09.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:09.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:09.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:09.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1085585048' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-82038-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:09.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[41654]: osdmap e98: 8 total, 8 up, 8 in 2024-02-05T07:53:09.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:09.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-82038-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:09.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:09.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:09.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:10.446 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-02-05T07:53:10.446 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [----------] Global test environment set-up. 2024-02-05T07:53:10.447 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-02-05T07:53:10.447 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-02-05T07:53:10.447 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5434 ms) 2024-02-05T07:53:10.447 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-02-05T07:53:10.447 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (24 ms) 2024-02-05T07:53:10.447 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-02-05T07:53:10.447 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (21020 ms) 2024-02-05T07:53:10.447 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-02-05T07:53:10.447 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: attempt 0 of 20 2024-02-05T07:53:10.447 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: failed to deregister: 2024-02-05T07:53:10.447 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: { 2024-02-05T07:53:10.447 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "epoch": 21, 2024-02-05T07:53:10.447 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "modified": "2024-02-05T07:53:07.307394+0000", 2024-02-05T07:53:10.447 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "services": { 2024-02-05T07:53:10.447 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "laundry": { 2024-02-05T07:53:10.448 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "daemons": { 2024-02-05T07:53:10.448 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "summary": "", 2024-02-05T07:53:10.448 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "close-test-pid82631": { 2024-02-05T07:53:10.448 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "start_epoch": 21, 2024-02-05T07:53:10.448 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "start_stamp": "2024-02-05T07:53:05.428809+0000", 2024-02-05T07:53:10.448 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "gid": 25313, 2024-02-05T07:53:10.448 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "addr": "172.21.15.131:0/3980763472", 2024-02-05T07:53:10.448 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "metadata": { 2024-02-05T07:53:10.448 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "arch": "x86_64", 2024-02-05T07:53:10.448 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "ceph_release": "squid", 2024-02-05T07:53:10.448 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev)", 2024-02-05T07:53:10.448 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "ceph_version_short": "19.0.0-1241-g164d3be4", 2024-02-05T07:53:10.448 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-05T07:53:10.448 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "distro": "centos", 2024-02-05T07:53:10.449 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-02-05T07:53:10.449 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "distro_version": "9", 2024-02-05T07:53:10.449 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "foo": "bar", 2024-02-05T07:53:10.449 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "hostname": "smithi131", 2024-02-05T07:53:10.449 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Wed Jan 31 22:52:48 UTC 2024", 2024-02-05T07:53:10.449 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "kernel_version": "5.14.0-416.el9.x86_64", 2024-02-05T07:53:10.449 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-05T07:53:10.449 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "mem_total_kb": "32488408", 2024-02-05T07:53:10.449 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "os": "Linux", 2024-02-05T07:53:10.449 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "this": "that" 2024-02-05T07:53:10.449 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: }, 2024-02-05T07:53:10.449 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "task_status": {} 2024-02-05T07:53:10.449 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: }, 2024-02-05T07:53:10.449 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "pid82631": { 2024-02-05T07:53:10.449 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "start_epoch": 20, 2024-02-05T07:53:10.450 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "start_stamp": "2024-02-05T07:53:04.419753+0000", 2024-02-05T07:53:10.450 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "gid": 24947, 2024-02-05T07:53:10.450 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "addr": "172.21.15.131:0/1988828350", 2024-02-05T07:53:10.450 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "metadata": { 2024-02-05T07:53:10.450 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "arch": "x86_64", 2024-02-05T07:53:10.450 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "ceph_release": "squid", 2024-02-05T07:53:10.450 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev)", 2024-02-05T07:53:10.450 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "ceph_version_short": "19.0.0-1241-g164d3be4", 2024-02-05T07:53:10.450 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-05T07:53:10.450 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "distro": "centos", 2024-02-05T07:53:10.450 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-02-05T07:53:10.450 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "distro_version": "9", 2024-02-05T07:53:10.450 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "foo": "bar", 2024-02-05T07:53:10.450 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "hostname": "smithi131", 2024-02-05T07:53:10.450 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Wed Jan 31 22:52:48 UTC 2024", 2024-02-05T07:53:10.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "kernel_version": "5.14.0-416.el9.x86_64", 2024-02-05T07:53:10.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-05T07:53:10.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "mem_total_kb": "32488408", 2024-02-05T07:53:10.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "os": "Linux", 2024-02-05T07:53:10.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "this": "that" 2024-02-05T07:53:10.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: }, 2024-02-05T07:53:10.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "task_status": {} 2024-02-05T07:53:10.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: }, 2024-02-05T07:53:10.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "pid82665": { 2024-02-05T07:53:10.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "start_epoch": 19, 2024-02-05T07:53:10.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "start_stamp": "2024-02-05T07:52:48.775762+0000", 2024-02-05T07:53:10.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "gid": 24998, 2024-02-05T07:53:10.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "addr": "172.21.15.131:0/3848745495", 2024-02-05T07:53:10.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "metadata": { 2024-02-05T07:53:10.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "arch": "x86_64", 2024-02-05T07:53:10.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "ceph_release": "squid", 2024-02-05T07:53:10.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev)", 2024-02-05T07:53:10.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "ceph_version_short": "19.0.0-1241-g164d3be4", 2024-02-05T07:53:10.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-05T07:53:10.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "distro": "centos", 2024-02-05T07:53:10.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-02-05T07:53:10.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "distro_version": "9", 2024-02-05T07:53:10.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "foo": "bar", 2024-02-05T07:53:10.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "hostname": "smithi131", 2024-02-05T07:53:10.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Wed Jan 31 22:52:48 UTC 2024", 2024-02-05T07:53:10.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "kernel_version": "5.14.0-416.el9.x86_64", 2024-02-05T07:53:10.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-05T07:53:10.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "mem_total_kb": "32488408", 2024-02-05T07:53:10.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "os": "Linux", 2024-02-05T07:53:10.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "this": "that" 2024-02-05T07:53:10.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: }, 2024-02-05T07:53:10.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "task_status": {} 2024-02-05T07:53:10.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: } 2024-02-05T07:53:10.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: } 2024-02-05T07:53:10.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: }, 2024-02-05T07:53:10.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "rgw": { 2024-02-05T07:53:10.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "daemons": { 2024-02-05T07:53:10.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "summary": "", 2024-02-05T07:53:10.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "14442": { 2024-02-05T07:53:10.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "start_epoch": 12, 2024-02-05T07:53:10.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "start_stamp": "2024-02-05T07:49:54.574451+0000", 2024-02-05T07:53:10.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "gid": 14442, 2024-02-05T07:53:10.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "addr": "172.21.15.131:0/2126927015", 2024-02-05T07:53:10.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "metadata": { 2024-02-05T07:53:10.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "arch": "x86_64", 2024-02-05T07:53:10.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "ceph_release": "squid", 2024-02-05T07:53:10.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev)", 2024-02-05T07:53:10.454 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "ceph_version_short": "19.0.0-1241-g164d3be4", 2024-02-05T07:53:10.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:10 smithi157 ceph-mon[33345]: Health check failed: 3 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-05T07:53:10.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-82038-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:10.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:10.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:10 smithi157 ceph-mon[33345]: osdmap e99: 8 total, 8 up, 8 in 2024-02-05T07:53:10.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3149099367' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:10.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/7666609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:10.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:10.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:10.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1925071152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi131-82437-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:10.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:10.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:10.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[41654]: Health check failed: 3 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-05T07:53:10.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-82038-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:10.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:10.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[41654]: osdmap e99: 8 total, 8 up, 8 in 2024-02-05T07:53:10.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3149099367' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:10.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/7666609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:10.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:10.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:10.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1925071152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi131-82437-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:10.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:10.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:10.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[29507]: Health check failed: 3 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-05T07:53:10.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-82038-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:10.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:10.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[29507]: osdmap e99: 8 total, 8 up, 8 in 2024-02-05T07:53:10.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3149099367' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:10.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/7666609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:10.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:10.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:10.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1925071152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi131-82437-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:10.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:10.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:11.646 INFO:tasks.workunit.client.0.smithi131.stdout: api_servi ec_list: Running main() from gmock_main.cc 2024-02-05T07:53:11.647 INFO:tasks.workunit.client.0.smithi131.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-02-05T07:53:11.647 INFO:tasks.workunit.client.0.smithi131.stdout: ec_list: [----------] Global test environment set-up. 2024-02-05T07:53:11.647 INFO:tasks.workunit.client.0.smithi131.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-02-05T07:53:11.647 INFO:tasks.workunit.client.0.smithi131.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-02-05T07:53:11.647 INFO:tasks.workunit.client.0.smithi131.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (15967 ms) 2024-02-05T07:53:11.647 INFO:tasks.workunit.client.0.smithi131.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-02-05T07:53:11.647 INFO:tasks.workunit.client.0.smithi131.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (8337 ms) 2024-02-05T07:53:11.647 INFO:tasks.workunit.client.0.smithi131.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-02-05T07:53:11.647 INFO:tasks.workunit.client.0.smithi131.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (8248 ms) 2024-02-05T07:53:11.647 INFO:tasks.workunit.client.0.smithi131.stdout: ec_list: [----------] 3 tests from NeoradosECList (32553 ms total) 2024-02-05T07:53:11.647 INFO:tasks.workunit.client.0.smithi131.stdout: ec_list: 2024-02-05T07:53:11.647 INFO:tasks.workunit.client.0.smithi131.stdout: ec_list: [----------] Global test environment tear-down 2024-02-05T07:53:11.647 INFO:tasks.workunit.client.0.smithi131.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (32553 ms total) 2024-02-05T07:53:11.647 INFO:tasks.workunit.client.0.smithi131.stdout: ec_list: [ PASSED ] 3 tests. 2024-02-05T07:53:11.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:11 smithi157 ceph-mon[33345]: pgmap v110: 616 pgs: 192 unknown, 424 active+clean; 145 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-05T07:53:11.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3241231597' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:53:11.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:11.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi131-83167-3"}]': finished 2024-02-05T07:53:11.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:11.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1925071152' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi131-82437-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:11.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3149099367' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:11.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:11 smithi157 ceph-mon[33345]: osdmap e100: 8 total, 8 up, 8 in 2024-02-05T07:53:11.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3186826943' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:11.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1612040208' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:11.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:11.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:11.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:11.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[41654]: pgmap v110: 616 pgs: 192 unknown, 424 active+clean; 145 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-05T07:53:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3241231597' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:53:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi131-83167-3"}]': finished 2024-02-05T07:53:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1925071152' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi131-82437-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3149099367' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[41654]: osdmap e100: 8 total, 8 up, 8 in 2024-02-05T07:53:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3186826943' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1612040208' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:11.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:11.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:11.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:11.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:11.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[29507]: pgmap v110: 616 pgs: 192 unknown, 424 active+clean; 145 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-05T07:53:11.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3241231597' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:53:11.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:11.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi131-83167-3"}]': finished 2024-02-05T07:53:11.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:11.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1925071152' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi131-82437-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:11.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3149099367' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:11.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[29507]: osdmap e100: 8 total, 8 up, 8 in 2024-02-05T07:53:11.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3186826943' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:11.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1612040208' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:11.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi131-83167-3"}]: dispatch 2024-02-05T07:53:11.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:11.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:11.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:12.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:12 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:53:12.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:12 smithi157 ceph-mon[33345]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:12.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:53:12.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi131-83167-3"}]': finished 2024-02-05T07:53:12.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:12.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:12.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-7"}]: dispatch 2024-02-05T07:53:12.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:12 smithi157 ceph-mon[33345]: osdmap e101: 8 total, 8 up, 8 in 2024-02-05T07:53:12.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/307398001' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-82038-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:12.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-82038-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:12.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-7"}]: dispatch 2024-02-05T07:53:12.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:12 smithi157 ceph-mon[33345]: pgmap v113: 680 pgs: 288 unknown, 392 active+clean; 145 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:53:12.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:12.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:53:12.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[29507]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:12.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:53:12.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi131-83167-3"}]': finished 2024-02-05T07:53:12.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:12.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:12.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-7"}]: dispatch 2024-02-05T07:53:12.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[29507]: osdmap e101: 8 total, 8 up, 8 in 2024-02-05T07:53:12.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/307398001' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-82038-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:12.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-82038-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:12.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-7"}]: dispatch 2024-02-05T07:53:12.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[29507]: pgmap v113: 680 pgs: 288 unknown, 392 active+clean; 145 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:53:12.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:12.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:53:12.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[41654]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:12.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:53:12.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi131-83167-3"}]': finished 2024-02-05T07:53:12.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:12.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:12.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-7"}]: dispatch 2024-02-05T07:53:12.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[41654]: osdmap e101: 8 total, 8 up, 8 in 2024-02-05T07:53:12.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/307398001' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-82038-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:12.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-82038-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:12.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-7"}]: dispatch 2024-02-05T07:53:12.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[41654]: pgmap v113: 680 pgs: 288 unknown, 392 active+clean; 145 MiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:53:12.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:53:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:53:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:53:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:13 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-82038-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:13 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-7"}]': finished 2024-02-05T07:53:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:13 smithi131 ceph-mon[41654]: osdmap e102: 8 total, 8 up, 8 in 2024-02-05T07:53:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:13 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2213233680' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:13 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:13 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:13 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:14.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:13 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-82038-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:14.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:13 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-7"}]': finished 2024-02-05T07:53:14.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:13 smithi131 ceph-mon[29507]: osdmap e102: 8 total, 8 up, 8 in 2024-02-05T07:53:14.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:13 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2213233680' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:14.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:13 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:14.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:13 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:14.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:13 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:13 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-82038-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:13 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-7"}]': finished 2024-02-05T07:53:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:13 smithi157 ceph-mon[33345]: osdmap e102: 8 total, 8 up, 8 in 2024-02-05T07:53:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:13 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2213233680' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:13 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:13 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:14.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:13 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:14.778 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: Running main() from gmock_main.cc 2024-02-05T07:53:14.778 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-02-05T07:53:14.779 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [----------] Global test environment set-up. 2024-02-05T07:53:14.779 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-02-05T07:53:14.779 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-02-05T07:53:14.779 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (10712 ms) 2024-02-05T07:53:14.779 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-02-05T07:53:14.779 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (6543 ms) 2024-02-05T07:53:14.779 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-02-05T07:53:14.779 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (4590 ms) 2024-02-05T07:53:14.779 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-02-05T07:53:14.779 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (3409 ms) 2024-02-05T07:53:14.779 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-02-05T07:53:14.779 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (3097 ms) 2024-02-05T07:53:14.779 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-02-05T07:53:14.779 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (3148 ms) 2024-02-05T07:53:14.780 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-02-05T07:53:14.780 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (4133 ms) 2024-02-05T07:53:14.780 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (35632 ms total) 2024-02-05T07:53:14.780 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: 2024-02-05T07:53:14.780 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [----------] Global test environment tear-down 2024-02-05T07:53:14.780 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (35632 ms total) 2024-02-05T07:53:14.780 INFO:tasks.workunit.client.0.smithi131.stdout: write_operations: [ PASSED ] 7 tests. 2024-02-05T07:53:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[41654]: osdmap e103: 8 total, 8 up, 8 in 2024-02-05T07:53:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3468117297' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[41654]: pgmap v116: 680 pgs: 64 creating+peering, 128 unknown, 488 active+clean; 145 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-02-05T07:53:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:53:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[41654]: osdmap e104: 8 total, 8 up, 8 in 2024-02-05T07:53:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7"}]: dispatch 2024-02-05T07:53:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7"}]: dispatch 2024-02-05T07:53:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/468712264' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-82038-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-82038-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:15.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:15.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[29507]: osdmap e103: 8 total, 8 up, 8 in 2024-02-05T07:53:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3468117297' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[29507]: pgmap v116: 680 pgs: 64 creating+peering, 128 unknown, 488 active+clean; 145 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-02-05T07:53:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:53:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[29507]: osdmap e104: 8 total, 8 up, 8 in 2024-02-05T07:53:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7"}]: dispatch 2024-02-05T07:53:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7"}]: dispatch 2024-02-05T07:53:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/468712264' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-82038-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:15.021 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-82038-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-82060-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:14 smithi157 ceph-mon[33345]: osdmap e103: 8 total, 8 up, 8 in 2024-02-05T07:53:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3468117297' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:14 smithi157 ceph-mon[33345]: pgmap v116: 680 pgs: 64 creating+peering, 128 unknown, 488 active+clean; 145 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-02-05T07:53:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:14 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:14 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82708-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:53:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:14 smithi157 ceph-mon[33345]: osdmap e104: 8 total, 8 up, 8 in 2024-02-05T07:53:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7"}]: dispatch 2024-02-05T07:53:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7"}]: dispatch 2024-02-05T07:53:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/468712264' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-82038-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-82038-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:15.493 INFO:tasks.workunit.client.0.smithi131.stdout:ce_pp: "container_hostname": "smithi131", 2024-02-05T07:53:15.493 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0", 2024-02-05T07:53:15.493 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-05T07:53:15.493 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "distro": "centos", 2024-02-05T07:53:15.493 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-02-05T07:53:15.493 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "distro_version": "8", 2024-02-05T07:53:15.494 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-02-05T07:53:15.494 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "frontend_type#0": "beast", 2024-02-05T07:53:15.494 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "hostname": "smithi131", 2024-02-05T07:53:15.494 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "id": "foo.a", 2024-02-05T07:53:15.494 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Wed Jan 31 22:52:48 UTC 2024", 2024-02-05T07:53:15.494 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "kernel_version": "5.14.0-416.el9.x86_64", 2024-02-05T07:53:15.494 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-05T07:53:15.494 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "mem_total_kb": "32488408", 2024-02-05T07:53:15.494 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "num_handles": "1", 2024-02-05T07:53:15.494 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "os": "Linux", 2024-02-05T07:53:15.494 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "pid": "7", 2024-02-05T07:53:15.494 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "realm_id": "", 2024-02-05T07:53:15.494 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "realm_name": "", 2024-02-05T07:53:15.494 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "zone_id": "0723b3b0-64ba-4e84-81f0-ff758d9cfc3e", 2024-02-05T07:53:15.495 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "zone_name": "default", 2024-02-05T07:53:15.495 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "zonegroup_id": "eaa855be-6eac-4a22-85a1-f6a07b1f9a5f", 2024-02-05T07:53:15.495 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "zonegroup_name": "default" 2024-02-05T07:53:15.495 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: }, 2024-02-05T07:53:15.495 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: "task_status": {} 2024-02-05T07:53:15.495 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: } 2024-02-05T07:53:15.495 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: } 2024-02-05T07:53:15.495 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: } 2024-02-05T07:53:15.495 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: } 2024-02-05T07:53:15.495 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: } 2024-02-05T07:53:15.495 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: 2024-02-05T07:53:15.495 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: attempt 1 of 20 2024-02-05T07:53:15.495 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (10088 ms) 2024-02-05T07:53:15.495 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (36566 ms total) 2024-02-05T07:53:15.495 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: 2024-02-05T07:53:15.496 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [----------] Global test environment tear-down 2024-02-05T07:53:15.496 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (36566 ms total) 2024-02-05T07:53:15.496 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-02-05T07:53:15.770 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-02-05T07:53:15.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-02-05T07:53:15.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-02-05T07:53:15.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-02-05T07:53:15.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-02-05T07:53:15.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-02-05T07:53:15.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-02-05T07:53:15.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: 2024-02-05T07:53:15.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-02-05T07:53:15.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-02-05T07:53:15.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (10917 ms) 2024-02-05T07:53:15.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-02-05T07:53:15.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (6541 ms) 2024-02-05T07:53:15.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-02-05T07:53:15.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (4609 ms) 2024-02-05T07:53:15.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-02-05T07:53:15.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (5502 ms) 2024-02-05T07:53:15.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-02-05T07:53:15.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3070 ms) 2024-02-05T07:53:15.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-02-05T07:53:15.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (3076 ms) 2024-02-05T07:53:15.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-02-05T07:53:15.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (3111 ms) 2024-02-05T07:53:15.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (36826 ms total) 2024-02-05T07:53:15.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: 2024-02-05T07:53:15.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-02-05T07:53:15.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (36827 ms total) 2024-02-05T07:53:15.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-02-05T07:53:16.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:16.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2908803084' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:53:16.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7"}]': finished 2024-02-05T07:53:16.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-82038-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:16.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:15 smithi131 ceph-mon[41654]: osdmap e105: 8 total, 8 up, 8 in 2024-02-05T07:53:16.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2564118651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi131-82060-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:16.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1879050902' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:16.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:16.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:16.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2908803084' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:53:16.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7"}]': finished 2024-02-05T07:53:16.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-82038-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:16.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:15 smithi131 ceph-mon[29507]: osdmap e105: 8 total, 8 up, 8 in 2024-02-05T07:53:16.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2564118651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi131-82060-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:16.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1879050902' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:16.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2908803084' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:53:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-7"}]': finished 2024-02-05T07:53:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-82038-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:15 smithi157 ceph-mon[33345]: osdmap e105: 8 total, 8 up, 8 in 2024-02-05T07:53:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2564118651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi131-82060-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1879050902' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:17.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:17 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:53:17.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:17 smithi131 ceph-mon[41654]: pgmap v119: 616 pgs: 32 creating+peering, 128 unknown, 456 active+clean; 145 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-02-05T07:53:17.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:17.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:17 smithi131 ceph-mon[41654]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:17.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2564118651' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi131-82060-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:17.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:17.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:17 smithi131 ceph-mon[41654]: osdmap e106: 8 total, 8 up, 8 in 2024-02-05T07:53:17.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:17 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:53:17.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:17 smithi131 ceph-mon[29507]: pgmap v119: 616 pgs: 32 creating+peering, 128 unknown, 456 active+clean; 145 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-02-05T07:53:17.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:17.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:17 smithi131 ceph-mon[29507]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:17.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2564118651' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi131-82060-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:17.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:17.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:17 smithi131 ceph-mon[29507]: osdmap e106: 8 total, 8 up, 8 in 2024-02-05T07:53:17.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:17 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:53:17.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:17 smithi157 ceph-mon[33345]: pgmap v119: 616 pgs: 32 creating+peering, 128 unknown, 456 active+clean; 145 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-02-05T07:53:17.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:17.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:17 smithi157 ceph-mon[33345]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:17.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2564118651' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi131-82060-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:17.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:17.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:17 smithi157 ceph-mon[33345]: osdmap e106: 8 total, 8 up, 8 in 2024-02-05T07:53:18.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:18.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[29507]: osdmap e107: 8 total, 8 up, 8 in 2024-02-05T07:53:18.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4292272595' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-82038-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:18.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-82038-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:18.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:18.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:18.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4"}]: dispatch 2024-02-05T07:53:18.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4"}]: dispatch 2024-02-05T07:53:18.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[29507]: pgmap v122: 680 pgs: 32 creating+peering, 224 unknown, 424 active+clean; 145 MiB data, 753 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:18.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:18.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:18.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[41654]: osdmap e107: 8 total, 8 up, 8 in 2024-02-05T07:53:18.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4292272595' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-82038-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:18.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-82038-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:18.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:18.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:18.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4"}]: dispatch 2024-02-05T07:53:18.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4"}]: dispatch 2024-02-05T07:53:18.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[41654]: pgmap v122: 680 pgs: 32 creating+peering, 224 unknown, 424 active+clean; 145 MiB data, 753 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:18.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:18.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:53:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:53:18.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:18.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:18 smithi157 ceph-mon[33345]: osdmap e107: 8 total, 8 up, 8 in 2024-02-05T07:53:18.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4292272595' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-82038-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:18.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-82038-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:18.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:18.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:18.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4"}]: dispatch 2024-02-05T07:53:18.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4"}]: dispatch 2024-02-05T07:53:18.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:18 smithi157 ceph-mon[33345]: pgmap v122: 680 pgs: 32 creating+peering, 224 unknown, 424 active+clean; 145 MiB data, 753 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:18.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:19.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:19.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-82038-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:19.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:19.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4"}]': finished 2024-02-05T07:53:19.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[41654]: osdmap e108: 8 total, 8 up, 8 in 2024-02-05T07:53:19.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6"}]: dispatch 2024-02-05T07:53:19.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2079394863' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-82060-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:19.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6"}]: dispatch 2024-02-05T07:53:19.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-82060-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:19.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:19.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:19.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:19.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:19.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-82038-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:19.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:19.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4"}]': finished 2024-02-05T07:53:19.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[29507]: osdmap e108: 8 total, 8 up, 8 in 2024-02-05T07:53:19.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6"}]: dispatch 2024-02-05T07:53:19.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2079394863' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-82060-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:19.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6"}]: dispatch 2024-02-05T07:53:19.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-82060-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:19.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:19.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:19.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:19.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:19 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:19.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-82038-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:19.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:19.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4"}]': finished 2024-02-05T07:53:19.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:19 smithi157 ceph-mon[33345]: osdmap e108: 8 total, 8 up, 8 in 2024-02-05T07:53:19.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6"}]: dispatch 2024-02-05T07:53:19.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2079394863' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-82060-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:19.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6"}]: dispatch 2024-02-05T07:53:19.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-82060-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:19.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:19.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:19.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:20.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:53:20.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6"}]': finished 2024-02-05T07:53:20.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-82060-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:20.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:53:20.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[41654]: osdmap e109: 8 total, 8 up, 8 in 2024-02-05T07:53:20.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-9"}]: dispatch 2024-02-05T07:53:20.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-9"}]: dispatch 2024-02-05T07:53:20.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi131-82531-6", "pool2": "test-rados-api-smithi131-82531-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T07:53:20.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi131-82531-6", "pool2": "test-rados-api-smithi131-82531-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T07:53:20.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[41654]: pgmap v125: 584 pgs: 32 creating+peering, 128 unknown, 424 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:53:20.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:20.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:53:20.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6"}]': finished 2024-02-05T07:53:20.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-82060-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:20.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:53:20.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[29507]: osdmap e109: 8 total, 8 up, 8 in 2024-02-05T07:53:20.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-9"}]: dispatch 2024-02-05T07:53:20.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-9"}]: dispatch 2024-02-05T07:53:20.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi131-82531-6", "pool2": "test-rados-api-smithi131-82531-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T07:53:20.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi131-82531-6", "pool2": "test-rados-api-smithi131-82531-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T07:53:20.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[29507]: pgmap v125: 584 pgs: 32 creating+peering, 128 unknown, 424 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:53:20.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:20 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:20.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:20 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:53:20.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:20 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-82531-4", "tierpool": "test-rados-api-smithi131-82531-6"}]': finished 2024-02-05T07:53:20.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:20 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-82060-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:20.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:20 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:53:20.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:20 smithi157 ceph-mon[33345]: osdmap e109: 8 total, 8 up, 8 in 2024-02-05T07:53:20.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:20 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-9"}]: dispatch 2024-02-05T07:53:20.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:20 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-9"}]: dispatch 2024-02-05T07:53:20.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:20 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1154934928' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi131-82531-6", "pool2": "test-rados-api-smithi131-82531-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T07:53:20.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:20 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi131-82531-6", "pool2": "test-rados-api-smithi131-82531-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T07:53:20.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:20 smithi157 ceph-mon[33345]: pgmap v125: 584 pgs: 32 creating+peering, 128 unknown, 424 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:53:20.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:20 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:21.501 INFO:tasks.workunit.client.0.smithi131.stdout:h_notify_pp: flushed 2024-02-05T07:53:21.501 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3004 ms) 2024-02-05T07:53:21.501 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-02-05T07:53:21.501 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: trying... 2024-02-05T07:53:21.501 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94027810016816 notify_id 420906795012 notifier_gid 25238 2024-02-05T07:53:21.501 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: timed out 2024-02-05T07:53:21.501 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: flushing 2024-02-05T07:53:21.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: flushed 2024-02-05T07:53:21.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3080 ms) 2024-02-05T07:53:21.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-02-05T07:53:21.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: List watches 2024-02-05T07:53:21.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: notify2 2024-02-05T07:53:21.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94027810016816 notify_id 433791696898 notifier_gid 25238 2024-02-05T07:53:21.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: notify2 done 2024-02-05T07:53:21.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: watch_check 2024-02-05T07:53:21.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: unwatch2 2024-02-05T07:53:21.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: flushing 2024-02-05T07:53:21.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: done 2024-02-05T07:53:21.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3170 ms) 2024-02-05T07:53:21.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-02-05T07:53:21.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: List watches 2024-02-05T07:53:21.502 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: notify2 2024-02-05T07:53:21.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94027810016816 notify_id 446676598789 notifier_gid 25238 2024-02-05T07:53:21.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: notify2 done 2024-02-05T07:53:21.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: watch_check 2024-02-05T07:53:21.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: unwatch2 2024-02-05T07:53:21.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: flushing 2024-02-05T07:53:21.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: done 2024-02-05T07:53:21.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3010 ms) 2024-02-05T07:53:21.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16836 ms total) 2024-02-05T07:53:21.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: 2024-02-05T07:53:21.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-02-05T07:53:21.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (42616 ms total) 2024-02-05T07:53:21.503 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-02-05T07:53:22.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-9"}]': finished 2024-02-05T07:53:22.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi131-82531-6", "pool2": "test-rados-api-smithi131-82531-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T07:53:22.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: osdmap e110: 8 total, 8 up, 8 in 2024-02-05T07:53:22.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-9", "mode": "writeback"}]: dispatch 2024-02-05T07:53:22.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3233156106' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:22.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-9", "mode": "writeback"}]: dispatch 2024-02-05T07:53:22.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2198430852' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi131-82038-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:22.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:22.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:22.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-02-05T07:53:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-02-05T07:53:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-02-05T07:53:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-02-05T07:53:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-02-05T07:53:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-02-05T07:53:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-02-05T07:53:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-02-05T07:53:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-02-05T07:53:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-02-05T07:53:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:53:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-9", "mode": "writeback"}]': finished 2024-02-05T07:53:22.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2198430852' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi131-82038-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:22.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:22.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:22.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: osdmap e111: 8 total, 8 up, 8 in 2024-02-05T07:53:22.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-82377-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2026227672' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-82060-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:22.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-82377-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-9"}]': finished 2024-02-05T07:53:22.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi131-82531-6", "pool2": "test-rados-api-smithi131-82531-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T07:53:22.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: osdmap e110: 8 total, 8 up, 8 in 2024-02-05T07:53:22.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-9", "mode": "writeback"}]: dispatch 2024-02-05T07:53:22.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3233156106' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:22.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-9", "mode": "writeback"}]: dispatch 2024-02-05T07:53:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2198430852' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi131-82038-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-02-05T07:53:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-02-05T07:53:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-02-05T07:53:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-02-05T07:53:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-02-05T07:53:22.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-02-05T07:53:22.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-02-05T07:53:22.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-02-05T07:53:22.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-02-05T07:53:22.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-02-05T07:53:22.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:22.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:53:22.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-9", "mode": "writeback"}]': finished 2024-02-05T07:53:22.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2198430852' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi131-82038-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:22.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:22.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:22.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: osdmap e111: 8 total, 8 up, 8 in 2024-02-05T07:53:22.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-82377-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2026227672' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-82060-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:22.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-82377-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-9"}]': finished 2024-02-05T07:53:22.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi131-82531-6", "pool2": "test-rados-api-smithi131-82531-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T07:53:22.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: osdmap e110: 8 total, 8 up, 8 in 2024-02-05T07:53:22.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-9", "mode": "writeback"}]: dispatch 2024-02-05T07:53:22.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3233156106' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:22.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-9", "mode": "writeback"}]: dispatch 2024-02-05T07:53:22.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2198430852' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi131-82038-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:22.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:22.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:22.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:22.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:22.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-02-05T07:53:22.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-02-05T07:53:22.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-02-05T07:53:22.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-02-05T07:53:22.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-02-05T07:53:22.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-02-05T07:53:22.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-02-05T07:53:22.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-02-05T07:53:22.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-02-05T07:53:22.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-02-05T07:53:22.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:22.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:53:22.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-9", "mode": "writeback"}]': finished 2024-02-05T07:53:22.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2198430852' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi131-82038-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:22.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:22.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:22.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: osdmap e111: 8 total, 8 up, 8 in 2024-02-05T07:53:22.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-82377-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:22.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2026227672' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-82060-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:22.273 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-82377-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:23.252 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:53:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:53:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:53:23.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-02-05T07:53:23.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-02-05T07:53:23.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-02-05T07:53:23.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-02-05T07:53:23.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-02-05T07:53:23.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-02-05T07:53:23.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-02-05T07:53:23.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-02-05T07:53:23.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-02-05T07:53:23.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-02-05T07:53:23.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: 12.3 deep-scrub starts 2024-02-05T07:53:23.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: 12.3 deep-scrub ok 2024-02-05T07:53:23.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:23.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: 12.8 deep-scrub starts 2024-02-05T07:53:23.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: 12.8 deep-scrub ok 2024-02-05T07:53:23.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-82060-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:23.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: 12.0 deep-scrub starts 2024-02-05T07:53:23.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: 12.0 deep-scrub ok 2024-02-05T07:53:23.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: 12.9 deep-scrub starts 2024-02-05T07:53:23.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: 12.9 deep-scrub ok 2024-02-05T07:53:23.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: 12.5 deep-scrub starts 2024-02-05T07:53:23.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: 12.5 deep-scrub ok 2024-02-05T07:53:23.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: pgmap v128: 548 pgs: 32 creating+peering, 160 unknown, 356 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:23.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: 12.6 deep-scrub starts 2024-02-05T07:53:23.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: 12.6 deep-scrub ok 2024-02-05T07:53:23.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:23.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:23.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-82060-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:23.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:53:23.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9"}]: dispatch 2024-02-05T07:53:23.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: osdmap e112: 8 total, 8 up, 8 in 2024-02-05T07:53:23.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9"}]: dispatch 2024-02-05T07:53:23.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-02-05T07:53:23.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-02-05T07:53:23.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-02-05T07:53:23.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-02-05T07:53:23.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-02-05T07:53:23.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-02-05T07:53:23.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-02-05T07:53:23.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-02-05T07:53:23.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-02-05T07:53:23.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-02-05T07:53:23.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: 12.3 deep-scrub starts 2024-02-05T07:53:23.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: 12.3 deep-scrub ok 2024-02-05T07:53:23.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:23.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: 12.8 deep-scrub starts 2024-02-05T07:53:23.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: 12.8 deep-scrub ok 2024-02-05T07:53:23.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-82060-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:23.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: 12.0 deep-scrub starts 2024-02-05T07:53:23.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: 12.0 deep-scrub ok 2024-02-05T07:53:23.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: 12.9 deep-scrub starts 2024-02-05T07:53:23.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: 12.9 deep-scrub ok 2024-02-05T07:53:23.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: 12.5 deep-scrub starts 2024-02-05T07:53:23.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: 12.5 deep-scrub ok 2024-02-05T07:53:23.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: pgmap v128: 548 pgs: 32 creating+peering, 160 unknown, 356 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:23.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: 12.6 deep-scrub starts 2024-02-05T07:53:23.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: 12.6 deep-scrub ok 2024-02-05T07:53:23.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:23.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:23.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-82060-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:23.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:53:23.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9"}]: dispatch 2024-02-05T07:53:23.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: osdmap e112: 8 total, 8 up, 8 in 2024-02-05T07:53:23.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9"}]: dispatch 2024-02-05T07:53:23.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-02-05T07:53:23.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-02-05T07:53:23.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-02-05T07:53:23.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-02-05T07:53:23.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-02-05T07:53:23.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-02-05T07:53:23.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-02-05T07:53:23.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-02-05T07:53:23.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-02-05T07:53:23.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-02-05T07:53:23.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: 12.3 deep-scrub starts 2024-02-05T07:53:23.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: 12.3 deep-scrub ok 2024-02-05T07:53:23.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:23.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: 12.8 deep-scrub starts 2024-02-05T07:53:23.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: 12.8 deep-scrub ok 2024-02-05T07:53:23.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-82060-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:23.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: 12.0 deep-scrub starts 2024-02-05T07:53:23.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: 12.0 deep-scrub ok 2024-02-05T07:53:23.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: 12.9 deep-scrub starts 2024-02-05T07:53:23.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: 12.9 deep-scrub ok 2024-02-05T07:53:23.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: 12.5 deep-scrub starts 2024-02-05T07:53:23.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: 12.5 deep-scrub ok 2024-02-05T07:53:23.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: pgmap v128: 548 pgs: 32 creating+peering, 160 unknown, 356 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:23.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: 12.6 deep-scrub starts 2024-02-05T07:53:23.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: 12.6 deep-scrub ok 2024-02-05T07:53:23.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:23.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:23.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-82060-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:23.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:53:23.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9"}]: dispatch 2024-02-05T07:53:23.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: osdmap e112: 8 total, 8 up, 8 in 2024-02-05T07:53:23.724 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9"}]: dispatch 2024-02-05T07:53:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[29507]: 12.2 deep-scrub starts 2024-02-05T07:53:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[29507]: 12.2 deep-scrub ok 2024-02-05T07:53:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[29507]: 12.4 deep-scrub starts 2024-02-05T07:53:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[29507]: 12.4 deep-scrub ok 2024-02-05T07:53:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[29507]: 12.7 deep-scrub starts 2024-02-05T07:53:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[29507]: 12.7 deep-scrub ok 2024-02-05T07:53:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:53:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-82377-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]': finished 2024-02-05T07:53:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9"}]': finished 2024-02-05T07:53:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[29507]: osdmap e113: 8 total, 8 up, 8 in 2024-02-05T07:53:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/309107449' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-82038-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-82038-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[29507]: pgmap v131: 556 pgs: 64 creating+peering, 72 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:24.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[41654]: 12.2 deep-scrub starts 2024-02-05T07:53:24.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[41654]: 12.2 deep-scrub ok 2024-02-05T07:53:24.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[41654]: 12.4 deep-scrub starts 2024-02-05T07:53:24.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[41654]: 12.4 deep-scrub ok 2024-02-05T07:53:24.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[41654]: 12.7 deep-scrub starts 2024-02-05T07:53:24.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[41654]: 12.7 deep-scrub ok 2024-02-05T07:53:24.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:24.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:53:24.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-82377-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]': finished 2024-02-05T07:53:24.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9"}]': finished 2024-02-05T07:53:24.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[41654]: osdmap e113: 8 total, 8 up, 8 in 2024-02-05T07:53:24.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/309107449' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-82038-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:24.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-82038-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:24.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[41654]: pgmap v131: 556 pgs: 64 creating+peering, 72 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:24.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:24 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:24.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:24 smithi157 ceph-mon[33345]: 12.2 deep-scrub starts 2024-02-05T07:53:24.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:24 smithi157 ceph-mon[33345]: 12.2 deep-scrub ok 2024-02-05T07:53:24.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:24 smithi157 ceph-mon[33345]: 12.4 deep-scrub starts 2024-02-05T07:53:24.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:24 smithi157 ceph-mon[33345]: 12.4 deep-scrub ok 2024-02-05T07:53:24.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:24 smithi157 ceph-mon[33345]: 12.7 deep-scrub starts 2024-02-05T07:53:24.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:24 smithi157 ceph-mon[33345]: 12.7 deep-scrub ok 2024-02-05T07:53:24.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:24 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:24.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:24 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:53:24.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-82377-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]': finished 2024-02-05T07:53:24.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-9"}]': finished 2024-02-05T07:53:24.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:24 smithi157 ceph-mon[33345]: osdmap e113: 8 total, 8 up, 8 in 2024-02-05T07:53:24.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:24 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/309107449' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-82038-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:24.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-82038-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:24.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:24 smithi157 ceph-mon[33345]: pgmap v131: 556 pgs: 64 creating+peering, 72 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:24.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:24 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:25.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-82038-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:25.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:25 smithi131 ceph-mon[41654]: osdmap e114: 8 total, 8 up, 8 in 2024-02-05T07:53:25.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1545550804' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:25.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:25.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:25.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-82038-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:25.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:25 smithi131 ceph-mon[29507]: osdmap e114: 8 total, 8 up, 8 in 2024-02-05T07:53:25.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1545550804' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:25.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:25.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:25.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-82038-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:25.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:25 smithi157 ceph-mon[33345]: osdmap e114: 8 total, 8 up, 8 in 2024-02-05T07:53:25.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1545550804' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:25.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:25.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:26 smithi157 ceph-mon[33345]: 12.1 deep-scrub starts 2024-02-05T07:53:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:26 smithi157 ceph-mon[33345]: 12.1 deep-scrub ok 2024-02-05T07:53:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:26 smithi157 ceph-mon[33345]: osdmap e115: 8 total, 8 up, 8 in 2024-02-05T07:53:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:26 smithi157 ceph-mon[33345]: pgmap v134: 556 pgs: 200 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-02-05T07:53:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:26 smithi157 ceph-mon[33345]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:26 smithi157 ceph-mon[33345]: osdmap e116: 8 total, 8 up, 8 in 2024-02-05T07:53:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[29507]: 12.1 deep-scrub starts 2024-02-05T07:53:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[29507]: 12.1 deep-scrub ok 2024-02-05T07:53:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[29507]: osdmap e115: 8 total, 8 up, 8 in 2024-02-05T07:53:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[29507]: pgmap v134: 556 pgs: 200 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-02-05T07:53:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:27.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[29507]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:27.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:27.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:27.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[29507]: osdmap e116: 8 total, 8 up, 8 in 2024-02-05T07:53:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[41654]: 12.1 deep-scrub starts 2024-02-05T07:53:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[41654]: 12.1 deep-scrub ok 2024-02-05T07:53:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[41654]: osdmap e115: 8 total, 8 up, 8 in 2024-02-05T07:53:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[41654]: pgmap v134: 556 pgs: 200 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-02-05T07:53:27.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:27.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[41654]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:27.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:27.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82302-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:27.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:26 smithi131 ceph-mon[41654]: osdmap e116: 8 total, 8 up, 8 in 2024-02-05T07:53:28.442 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3246316758' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-82038-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:28.442 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-82038-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:28.442 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:28.442 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-82038-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:28.442 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:28 smithi157 ceph-mon[33345]: osdmap e117: 8 total, 8 up, 8 in 2024-02-05T07:53:28.443 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82302-10", "tierpool":"test-rados-api-smithi131-82302-10-cache", "force_nonempty":""}]: dispatch 2024-02-05T07:53:28.443 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3899160908' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-82060-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:28.443 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-82060-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:28.508 INFO:tasks.workunit.client.0.smithi131.stdout: misc: Running main() from gmock_main.cc 2024-02-05T07:53:28.508 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-02-05T07:53:28.508 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [----------] Global test environment set-up. 2024-02-05T07:53:28.508 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-02-05T07:53:28.508 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-02-05T07:53:28.508 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ OK ] NeoRadosMisc.Version (9494 ms) 2024-02-05T07:53:28.508 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-02-05T07:53:28.509 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (5412 ms) 2024-02-05T07:53:28.509 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-02-05T07:53:28.509 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ OK ] NeoRadosMisc.LongName (5862 ms) 2024-02-05T07:53:28.509 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-02-05T07:53:28.509 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (4492 ms) 2024-02-05T07:53:28.509 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-02-05T07:53:28.509 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (3100 ms) 2024-02-05T07:53:28.509 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-02-05T07:53:28.509 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (3141 ms) 2024-02-05T07:53:28.509 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-02-05T07:53:28.509 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ OK ] NeoRadosMisc.Exec (3106 ms) 2024-02-05T07:53:28.509 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-02-05T07:53:28.509 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (2760 ms) 2024-02-05T07:53:28.509 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-02-05T07:53:28.509 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (3015 ms) 2024-02-05T07:53:28.509 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-02-05T07:53:28.510 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ OK ] NeoRadosMisc.BigObject (3234 ms) 2024-02-05T07:53:28.510 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-02-05T07:53:28.510 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (1759 ms) 2024-02-05T07:53:28.510 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-02-05T07:53:28.510 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (3994 ms) 2024-02-05T07:53:28.510 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [----------] 12 tests from NeoRadosMisc (49370 ms total) 2024-02-05T07:53:28.510 INFO:tasks.workunit.client.0.smithi131.stdout: misc: 2024-02-05T07:53:28.510 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [----------] Global test environment tear-down 2024-02-05T07:53:28.510 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [==========] 12 tests from 1 test suite ran. (49370 ms total) 2024-02-05T07:53:28.510 INFO:tasks.workunit.client.0.smithi131.stdout: misc: [ PASSED ] 12 tests. 2024-02-05T07:53:28.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3246316758' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-82038-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:28.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-82038-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:28.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:28.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-82038-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:28.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:28 smithi131 ceph-mon[41654]: osdmap e117: 8 total, 8 up, 8 in 2024-02-05T07:53:28.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82302-10", "tierpool":"test-rados-api-smithi131-82302-10-cache", "force_nonempty":""}]: dispatch 2024-02-05T07:53:28.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3899160908' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-82060-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:28.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-82060-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:28.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3246316758' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-82038-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:28.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-82038-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:28.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:28.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-82038-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:28.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:28 smithi131 ceph-mon[29507]: osdmap e117: 8 total, 8 up, 8 in 2024-02-05T07:53:28.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82302-10", "tierpool":"test-rados-api-smithi131-82302-10-cache", "force_nonempty":""}]: dispatch 2024-02-05T07:53:28.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3899160908' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-82060-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:28.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-82060-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:28.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:53:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:53:29.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:29.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:29 smithi157 ceph-mon[33345]: pgmap v137: 588 pgs: 232 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:29.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:29.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:29.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:29 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:29.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82302-10", "tierpool":"test-rados-api-smithi131-82302-10-cache", "force_nonempty":""}]': finished 2024-02-05T07:53:29.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-82060-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:29.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:53:29.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:29 smithi157 ceph-mon[33345]: osdmap e118: 8 total, 8 up, 8 in 2024-02-05T07:53:29.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82302-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:29.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-11"}]: dispatch 2024-02-05T07:53:29.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-11"}]: dispatch 2024-02-05T07:53:29.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:29.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[41654]: pgmap v137: 588 pgs: 232 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:29.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:29.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:29.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:29.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82302-10", "tierpool":"test-rados-api-smithi131-82302-10-cache", "force_nonempty":""}]': finished 2024-02-05T07:53:29.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-82060-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:29.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:53:29.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[41654]: osdmap e118: 8 total, 8 up, 8 in 2024-02-05T07:53:29.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82302-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:29.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-11"}]: dispatch 2024-02-05T07:53:29.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-11"}]: dispatch 2024-02-05T07:53:29.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:29.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[29507]: pgmap v137: 588 pgs: 232 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:29.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:29.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:29.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:29.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82302-10", "tierpool":"test-rados-api-smithi131-82302-10-cache", "force_nonempty":""}]': finished 2024-02-05T07:53:29.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-82060-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:29.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:53:29.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[29507]: osdmap e118: 8 total, 8 up, 8 in 2024-02-05T07:53:29.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82302-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:29.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-11"}]: dispatch 2024-02-05T07:53:29.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-11"}]: dispatch 2024-02-05T07:53:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82302-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-11"}]': finished 2024-02-05T07:53:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[41654]: osdmap e119: 8 total, 8 up, 8 in 2024-02-05T07:53:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-11", "mode": "writeback"}]: dispatch 2024-02-05T07:53:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-11", "mode": "writeback"}]: dispatch 2024-02-05T07:53:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82302-10", "tierpool":"test-rados-api-smithi131-82302-10-cache"}]: dispatch 2024-02-05T07:53:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1311116923' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi131-82038-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82302-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-11"}]': finished 2024-02-05T07:53:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[29507]: osdmap e119: 8 total, 8 up, 8 in 2024-02-05T07:53:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-11", "mode": "writeback"}]: dispatch 2024-02-05T07:53:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-11", "mode": "writeback"}]: dispatch 2024-02-05T07:53:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82302-10", "tierpool":"test-rados-api-smithi131-82302-10-cache"}]: dispatch 2024-02-05T07:53:30.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1311116923' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi131-82038-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:30.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:30.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:30 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:30.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:30 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:30.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82302-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:30.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-11"}]': finished 2024-02-05T07:53:30.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:30 smithi157 ceph-mon[33345]: osdmap e119: 8 total, 8 up, 8 in 2024-02-05T07:53:30.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-11", "mode": "writeback"}]: dispatch 2024-02-05T07:53:30.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-11", "mode": "writeback"}]: dispatch 2024-02-05T07:53:30.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82302-10", "tierpool":"test-rados-api-smithi131-82302-10-cache"}]: dispatch 2024-02-05T07:53:30.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1311116923' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi131-82038-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:31 smithi157 ceph-mon[33345]: pgmap v140: 556 pgs: 32 creating+peering, 96 unknown, 428 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-02-05T07:53:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:31 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:31 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:53:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:31 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-11", "mode": "writeback"}]': finished 2024-02-05T07:53:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:31 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82302-10", "tierpool":"test-rados-api-smithi131-82302-10-cache"}]': finished 2024-02-05T07:53:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:31 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1311116923' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi131-82038-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:31 smithi157 ceph-mon[33345]: osdmap e120: 8 total, 8 up, 8 in 2024-02-05T07:53:31.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:31 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4084817132' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-82060-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:31.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:31 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-82060-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[41654]: pgmap v140: 556 pgs: 32 creating+peering, 96 unknown, 428 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-02-05T07:53:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:53:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-11", "mode": "writeback"}]': finished 2024-02-05T07:53:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82302-10", "tierpool":"test-rados-api-smithi131-82302-10-cache"}]': finished 2024-02-05T07:53:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1311116923' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi131-82038-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[41654]: osdmap e120: 8 total, 8 up, 8 in 2024-02-05T07:53:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4084817132' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-82060-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-82060-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[29507]: pgmap v140: 556 pgs: 32 creating+peering, 96 unknown, 428 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-02-05T07:53:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:53:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-11", "mode": "writeback"}]': finished 2024-02-05T07:53:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2525501809' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82302-10", "tierpool":"test-rados-api-smithi131-82302-10-cache"}]': finished 2024-02-05T07:53:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1311116923' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi131-82038-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[29507]: osdmap e120: 8 total, 8 up, 8 in 2024-02-05T07:53:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4084817132' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-82060-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:31 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-82060-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:32 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:32 smithi157 ceph-mon[33345]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-82060-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:32 smithi157 ceph-mon[33345]: osdmap e121: 8 total, 8 up, 8 in 2024-02-05T07:53:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:32 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:32.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:32.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:32 smithi157 ceph-mon[33345]: pgmap v143: 556 pgs: 128 unknown, 428 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-02-05T07:53:32.516 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: Running main() from gmock_main.cc 2024-02-05T07:53:32.516 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-02-05T07:53:32.516 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [----------] Global test environment set-up. 2024-02-05T07:53:32.517 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-02-05T07:53:32.517 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-02-05T07:53:32.517 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolList (11062 ms) 2024-02-05T07:53:32.517 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-02-05T07:53:32.517 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (6539 ms) 2024-02-05T07:53:32.517 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-02-05T07:53:32.517 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (4620 ms) 2024-02-05T07:53:32.517 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-02-05T07:53:32.517 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (4705 ms) 2024-02-05T07:53:32.517 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-02-05T07:53:32.517 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3864 ms) 2024-02-05T07:53:32.517 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-02-05T07:53:32.517 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5151 ms) 2024-02-05T07:53:32.517 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-02-05T07:53:32.517 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (4774 ms) 2024-02-05T07:53:32.518 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-02-05T07:53:32.518 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (5008 ms) 2024-02-05T07:53:32.518 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-02-05T07:53:32.518 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (7994 ms) 2024-02-05T07:53:32.518 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [----------] 9 tests from LibRadosPools (53717 ms total) 2024-02-05T07:53:32.518 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: 2024-02-05T07:53:32.518 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [----------] Global test environment tear-down 2024-02-05T07:53:32.518 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (53717 ms total) 2024-02-05T07:53:32.518 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ PASSED ] 9 tests. 2024-02-05T07:53:32.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:32 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:32.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:32 smithi131 ceph-mon[41654]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:32.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-82060-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:32.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:32 smithi131 ceph-mon[41654]: osdmap e121: 8 total, 8 up, 8 in 2024-02-05T07:53:32.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:32 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:32.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:32.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:32 smithi131 ceph-mon[41654]: pgmap v143: 556 pgs: 128 unknown, 428 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-02-05T07:53:32.523 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:32 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:32.524 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:32 smithi131 ceph-mon[29507]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:32.524 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-82060-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:32.524 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:32 smithi131 ceph-mon[29507]: osdmap e121: 8 total, 8 up, 8 in 2024-02-05T07:53:32.524 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:32 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:32.524 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:53:32.524 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:32 smithi131 ceph-mon[29507]: pgmap v143: 556 pgs: 128 unknown, 428 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-02-05T07:53:33.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:53:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:53:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:53:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:33 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:53:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:33 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11"}]: dispatch 2024-02-05T07:53:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:33 smithi131 ceph-mon[29507]: osdmap e122: 8 total, 8 up, 8 in 2024-02-05T07:53:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:33 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2676075268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi131-82038-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11"}]: dispatch 2024-02-05T07:53:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:33 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:53:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:33 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11"}]: dispatch 2024-02-05T07:53:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:33 smithi131 ceph-mon[41654]: osdmap e122: 8 total, 8 up, 8 in 2024-02-05T07:53:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:33 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2676075268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi131-82038-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11"}]: dispatch 2024-02-05T07:53:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:33 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:53:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:33 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11"}]: dispatch 2024-02-05T07:53:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:33 smithi157 ceph-mon[33345]: osdmap e122: 8 total, 8 up, 8 in 2024-02-05T07:53:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:33 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2676075268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi131-82038-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11"}]: dispatch 2024-02-05T07:53:33.510 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: Running main() from gmock_main.cc 2024-02-05T07:53:33.510 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-02-05T07:53:33.510 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [----------] Global test environment set-up. 2024-02-05T07:53:33.510 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-02-05T07:53:33.510 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-02-05T07:53:33.510 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (9488 ms) 2024-02-05T07:53:33.510 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-02-05T07:53:33.510 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (6709 ms) 2024-02-05T07:53:33.510 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-02-05T07:53:33.510 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (4565 ms) 2024-02-05T07:53:33.511 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-02-05T07:53:33.511 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (3489 ms) 2024-02-05T07:53:33.511 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-02-05T07:53:33.511 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (3110 ms) 2024-02-05T07:53:33.511 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-02-05T07:53:33.511 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (3078 ms) 2024-02-05T07:53:33.511 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-02-05T07:53:33.511 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (3073 ms) 2024-02-05T07:53:33.511 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-02-05T07:53:33.511 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (3088 ms) 2024-02-05T07:53:33.511 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-02-05T07:53:33.511 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (2767 ms) 2024-02-05T07:53:33.511 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-02-05T07:53:33.512 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (2990 ms) 2024-02-05T07:53:33.512 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-02-05T07:53:33.512 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (3018 ms) 2024-02-05T07:53:33.512 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-02-05T07:53:33.512 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (2996 ms) 2024-02-05T07:53:33.512 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-02-05T07:53:33.512 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (3010 ms) 2024-02-05T07:53:33.512 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-02-05T07:53:33.512 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (2985 ms) 2024-02-05T07:53:33.512 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (54366 ms total) 2024-02-05T07:53:33.512 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: 2024-02-05T07:53:33.512 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [----------] Global test environment tear-down 2024-02-05T07:53:33.512 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (54366 ms total) 2024-02-05T07:53:33.513 INFO:tasks.workunit.client.0.smithi131.stdout: read_operations: [ PASSED ] 14 tests. 2024-02-05T07:53:33.513 INFO:tasks.workunit.client.0.smithi131.stdout: io: Running main() from gmock_main.cc 2024-02-05T07:53:33.513 INFO:tasks.workunit.client.0.smithi131.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-02-05T07:53:33.513 INFO:tasks.workunit.client.0.smithi131.stdout: io: [----------] Global test environment set-up. 2024-02-05T07:53:33.513 INFO:tasks.workunit.client.0.smithi131.stdout: io: [----------] 14 tests from NeoRadosIo 2024-02-05T07:53:33.513 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-02-05T07:53:33.513 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ OK ] NeoRadosIo.Limits (9551 ms) 2024-02-05T07:53:33.513 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-02-05T07:53:33.513 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (6707 ms) 2024-02-05T07:53:33.514 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-02-05T07:53:33.514 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ OK ] NeoRadosIo.ReadOp (4591 ms) 2024-02-05T07:53:33.514 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-02-05T07:53:33.514 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ OK ] NeoRadosIo.SparseRead (2422 ms) 2024-02-05T07:53:33.514 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-02-05T07:53:33.514 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ OK ] NeoRadosIo.RoundTrip (3365 ms) 2024-02-05T07:53:33.514 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-02-05T07:53:33.514 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (3860 ms) 2024-02-05T07:53:33.514 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-02-05T07:53:33.514 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (3086 ms) 2024-02-05T07:53:33.514 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-02-05T07:53:33.514 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (3080 ms) 2024-02-05T07:53:33.514 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-02-05T07:53:33.514 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (2771 ms) 2024-02-05T07:53:33.514 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-02-05T07:53:33.515 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ OK ] NeoRadosIo.Trunc (2982 ms) 2024-02-05T07:53:33.515 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-02-05T07:53:33.515 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ OK ] NeoRadosIo.Remove (3016 ms) 2024-02-05T07:53:33.515 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-02-05T07:53:33.515 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (2998 ms) 2024-02-05T07:53:33.515 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-02-05T07:53:33.515 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ OK ] NeoRadosIo.RmXattr (3010 ms) 2024-02-05T07:53:33.515 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-02-05T07:53:33.515 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ OK ] NeoRadosIo.GetXattrs (2988 ms) 2024-02-05T07:53:33.515 INFO:tasks.workunit.client.0.smithi131.stdout: io: [----------] 14 tests from NeoRadosIo (54427 ms total) 2024-02-05T07:53:33.515 INFO:tasks.workunit.client.0.smithi131.stdout: io: 2024-02-05T07:53:33.515 INFO:tasks.workunit.client.0.smithi131.stdout: io: [----------] Global test environment tear-down 2024-02-05T07:53:33.515 INFO:tasks.workunit.client.0.smithi131.stdout: io: [==========] 14 tests from 1 test suite ran. (54427 ms total) 2024-02-05T07:53:33.515 INFO:tasks.workunit.client.0.smithi131.stdout: io: [ PASSED ] 14 tests. 2024-02-05T07:53:34.417 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:34.417 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:34 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:53:34.417 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2676075268' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi131-82038-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:34.418 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11"}]': finished 2024-02-05T07:53:34.418 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:34 smithi157 ceph-mon[33345]: osdmap e123: 8 total, 8 up, 8 in 2024-02-05T07:53:34.418 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:34.418 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:34.418 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1516591524' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi131-82060-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:34.418 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi131-82060-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:34.418 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:34 smithi157 ceph-mon[33345]: pgmap v146: 484 pgs: 32 creating+peering, 32 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-02-05T07:53:34.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:34.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:53:34.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2676075268' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi131-82038-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:34.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11"}]': finished 2024-02-05T07:53:34.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[41654]: osdmap e123: 8 total, 8 up, 8 in 2024-02-05T07:53:34.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:34.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:34.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1516591524' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi131-82060-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:34.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi131-82060-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:34.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[41654]: pgmap v146: 484 pgs: 32 creating+peering, 32 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-02-05T07:53:34.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:34.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:53:34.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2676075268' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi131-82038-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:34.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-11"}]': finished 2024-02-05T07:53:34.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[29507]: osdmap e123: 8 total, 8 up, 8 in 2024-02-05T07:53:34.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:34.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:34.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1516591524' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi131-82060-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:34.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi131-82060-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:34.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:34 smithi131 ceph-mon[29507]: pgmap v146: 484 pgs: 32 creating+peering, 32 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-02-05T07:53:35.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:35.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]': finished 2024-02-05T07:53:35.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi131-82060-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:35.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:35 smithi157 ceph-mon[33345]: osdmap e124: 8 total, 8 up, 8 in 2024-02-05T07:53:35.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:35.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:35.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:35.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]': finished 2024-02-05T07:53:35.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi131-82060-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:35.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:35 smithi131 ceph-mon[29507]: osdmap e124: 8 total, 8 up, 8 in 2024-02-05T07:53:35.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:35.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:35.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:35.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-82377-10"}]': finished 2024-02-05T07:53:35.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi131-82060-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:35.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:35 smithi131 ceph-mon[41654]: osdmap e124: 8 total, 8 up, 8 in 2024-02-05T07:53:35.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2239568768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:35.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-82377-10"}]: dispatch 2024-02-05T07:53:36.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:36.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-82377-10"}]': finished 2024-02-05T07:53:36.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:36 smithi157 ceph-mon[33345]: osdmap e125: 8 total, 8 up, 8 in 2024-02-05T07:53:36.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:36.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1709585853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi131-82038-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:36.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:36.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi131-82038-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:36.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]: dispatch 2024-02-05T07:53:36.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]: dispatch 2024-02-05T07:53:36.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:36.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:36 smithi157 ceph-mon[33345]: pgmap v149: 452 pgs: 64 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-02-05T07:53:36.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:36.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-82377-10"}]': finished 2024-02-05T07:53:36.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[29507]: osdmap e125: 8 total, 8 up, 8 in 2024-02-05T07:53:36.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:36.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1709585853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi131-82038-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:36.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:36.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi131-82038-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:36.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]: dispatch 2024-02-05T07:53:36.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]: dispatch 2024-02-05T07:53:36.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:36.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[29507]: pgmap v149: 452 pgs: 64 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-02-05T07:53:36.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:36.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-82377-10"}]': finished 2024-02-05T07:53:36.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[41654]: osdmap e125: 8 total, 8 up, 8 in 2024-02-05T07:53:36.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:36.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1709585853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi131-82038-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:36.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:36.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi131-82038-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:36.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]: dispatch 2024-02-05T07:53:36.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]: dispatch 2024-02-05T07:53:36.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:36.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:36 smithi131 ceph-mon[41654]: pgmap v149: 452 pgs: 64 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-02-05T07:53:37.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:37.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:37 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:37.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:37.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi131-82038-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:37.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:37.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:37 smithi157 ceph-mon[33345]: osdmap e126: 8 total, 8 up, 8 in 2024-02-05T07:53:37.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2533910926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi131-82060-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:37.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi131-82377-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]: dispatch 2024-02-05T07:53:37.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi131-82060-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:37.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:37.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:37.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:37.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi131-82038-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:37.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:37.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[41654]: osdmap e126: 8 total, 8 up, 8 in 2024-02-05T07:53:37.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2533910926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi131-82060-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:37.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi131-82377-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]: dispatch 2024-02-05T07:53:37.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi131-82060-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:37.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:37.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:37.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:37.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi131-82038-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:37.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:37.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[29507]: osdmap e126: 8 total, 8 up, 8 in 2024-02-05T07:53:37.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2533910926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi131-82060-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:37.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi131-82377-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]: dispatch 2024-02-05T07:53:37.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi131-82060-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:38.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:53:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:53:38.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:38 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:38.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi131-82060-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:38.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:38 smithi157 ceph-mon[33345]: osdmap e127: 8 total, 8 up, 8 in 2024-02-05T07:53:38.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:38 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:38.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi131-82060-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:38.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:38 smithi131 ceph-mon[41654]: osdmap e127: 8 total, 8 up, 8 in 2024-02-05T07:53:38.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:38 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:38.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi131-82060-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:38.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:38 smithi131 ceph-mon[29507]: osdmap e127: 8 total, 8 up, 8 in 2024-02-05T07:53:39.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:39 smithi157 ceph-mon[33345]: pgmap v152: 420 pgs: 64 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:39.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:39 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:39.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:39 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:39.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:39 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi131-82377-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]': finished 2024-02-05T07:53:39.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:39 smithi157 ceph-mon[33345]: osdmap e128: 8 total, 8 up, 8 in 2024-02-05T07:53:39.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:39 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/99871918' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi131-82038-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:39.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi131-82038-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:39.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:39 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:39.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:39 smithi131 ceph-mon[41654]: pgmap v152: 420 pgs: 64 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:39.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:39 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:39.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:39 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:39.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:39 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi131-82377-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]': finished 2024-02-05T07:53:39.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:39 smithi131 ceph-mon[41654]: osdmap e128: 8 total, 8 up, 8 in 2024-02-05T07:53:39.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:39 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/99871918' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi131-82038-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:39.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi131-82038-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:39.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:39 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:39.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:39 smithi131 ceph-mon[29507]: pgmap v152: 420 pgs: 64 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:39.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:39.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:39 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:39.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi131-82377-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]': finished 2024-02-05T07:53:39.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:39 smithi131 ceph-mon[29507]: osdmap e128: 8 total, 8 up, 8 in 2024-02-05T07:53:39.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/99871918' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi131-82038-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:39.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi131-82038-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:39.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:40.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-02-05T07:53:40.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] Global test environment set-up. 2024-02-05T07:53:40.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-02-05T07:53:40.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-02-05T07:53:40.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-02-05T07:53:40.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-05T07:53:40.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: 2024-02-05T07:53:40.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-02-05T07:53:40.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-02-05T07:53:40.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-02-05T07:53:40.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: 2024-02-05T07:52:38.836+0000 7fbdb1c35780 -1 failed for service _ceph-mon._tcp 2024-02-05T07:53:40.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: 2024-02-05T07:52:38.836+0000 7fbdb1c35780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-02-05T07:53:40.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (61 ms) 2024-02-05T07:53:40.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-02-05T07:53:40.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2020 ms) 2024-02-05T07:53:40.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2081 ms total) 2024-02-05T07:53:40.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: 2024-02-05T07:53:40.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-02-05T07:53:40.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-02-05T07:53:40.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559250cfa720 2024-02-05T07:53:40.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510221b0 2024-02-05T07:53:40.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251022b00 2024-02-05T07:53:40.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510230f0 2024-02-05T07:53:40.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251022fe0 2024-02-05T07:53:40.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251023600 2024-02-05T07:53:40.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251023aa0 2024-02-05T07:53:40.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251023fc0 2024-02-05T07:53:40.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510244c0 2024-02-05T07:53:40.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251024930 2024-02-05T07:53:40.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251024a40 2024-02-05T07:53:40.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251024ba0 2024-02-05T07:53:40.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510245d0 2024-02-05T07:53:40.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251024760 2024-02-05T07:53:40.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251025190 2024-02-05T07:53:40.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251024cb0 2024-02-05T07:53:40.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251024dc0 2024-02-05T07:53:40.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251024f20 2024-02-05T07:53:40.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510252a0 2024-02-05T07:53:40.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251025760 2024-02-05T07:53:40.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251025870 2024-02-05T07:53:40.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251025e20 2024-02-05T07:53:40.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510253b0 2024-02-05T07:53:40.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251025510 2024-02-05T07:53:40.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251025980 2024-02-05T07:53:40.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251025a90 2024-02-05T07:53:40.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251025ba0 2024-02-05T07:53:40.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251025cb0 2024-02-05T07:53:40.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510263d0 2024-02-05T07:53:40.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251025f30 2024-02-05T07:53:40.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251026060 2024-02-05T07:53:40.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251026170 2024-02-05T07:53:40.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510262c0 2024-02-05T07:53:40.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251026980 2024-02-05T07:53:40.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510264e0 2024-02-05T07:53:40.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510265f0 2024-02-05T07:53:40.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251026720 2024-02-05T07:53:40.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251026850 2024-02-05T07:53:40.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251026f70 2024-02-05T07:53:40.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251026ad0 2024-02-05T07:53:40.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251026c00 2024-02-05T07:53:40.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251026d10 2024-02-05T07:53:40.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251026e60 2024-02-05T07:53:40.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251027520 2024-02-05T07:53:40.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251027080 2024-02-05T07:53:40.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251027190 2024-02-05T07:53:40.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510272a0 2024-02-05T07:53:40.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510273b0 2024-02-05T07:53:40.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251027af0 2024-02-05T07:53:40.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251027650 2024-02-05T07:53:40.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251027760 2024-02-05T07:53:40.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi131-82038-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:40.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:40 smithi131 ceph-mon[41654]: osdmap e129: 8 total, 8 up, 8 in 2024-02-05T07:53:40.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:40 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/378530302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-82060-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:40.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-82060-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:40.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:40 smithi131 ceph-mon[41654]: pgmap v155: 492 pgs: 104 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 47 KiB/s wr, 48 op/s 2024-02-05T07:53:40.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:40 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:40.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi131-82038-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:40.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:40 smithi131 ceph-mon[29507]: osdmap e129: 8 total, 8 up, 8 in 2024-02-05T07:53:40.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:40 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/378530302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-82060-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:40.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-82060-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:40.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:40 smithi131 ceph-mon[29507]: pgmap v155: 492 pgs: 104 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 47 KiB/s wr, 48 op/s 2024-02-05T07:53:40.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:40 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:40.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi131-82038-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:40.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:40 smithi157 ceph-mon[33345]: osdmap e129: 8 total, 8 up, 8 in 2024-02-05T07:53:40.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:40 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/378530302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-82060-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:40.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-82060-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:40.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:40 smithi157 ceph-mon[33345]: pgmap v155: 492 pgs: 104 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 47 KiB/s wr, 48 op/s 2024-02-05T07:53:40.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:40 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-82060-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:41 smithi157 ceph-mon[33345]: osdmap e130: 8 total, 8 up, 8 in 2024-02-05T07:53:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:41.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:41 smithi157 ceph-mon[33345]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:41.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:53:41.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:41 smithi157 ceph-mon[33345]: osdmap e131: 8 total, 8 up, 8 in 2024-02-05T07:53:41.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-13"}]: dispatch 2024-02-05T07:53:41.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-13"}]: dispatch 2024-02-05T07:53:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-82060-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[29507]: osdmap e130: 8 total, 8 up, 8 in 2024-02-05T07:53:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[29507]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:53:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[29507]: osdmap e131: 8 total, 8 up, 8 in 2024-02-05T07:53:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-13"}]: dispatch 2024-02-05T07:53:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-13"}]: dispatch 2024-02-05T07:53:42.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-82060-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:42.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[41654]: osdmap e130: 8 total, 8 up, 8 in 2024-02-05T07:53:42.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:42.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:53:42.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:42.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[41654]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:42.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:53:42.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[41654]: osdmap e131: 8 total, 8 up, 8 in 2024-02-05T07:53:42.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-13"}]: dispatch 2024-02-05T07:53:42.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-13"}]: dispatch 2024-02-05T07:53:42.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1530541454' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-82038-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:42.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-82038-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:42.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[41654]: pgmap v158: 460 pgs: 72 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-02-05T07:53:42.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:42.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-13"}]': finished 2024-02-05T07:53:42.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-82038-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:42.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-13", "mode": "writeback"}]: dispatch 2024-02-05T07:53:42.944 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[41654]: osdmap e132: 8 total, 8 up, 8 in 2024-02-05T07:53:42.944 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-13", "mode": "writeback"}]: dispatch 2024-02-05T07:53:42.944 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2620189832' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi131-82060-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:42.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1530541454' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-82038-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:42.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-82038-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:42.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[29507]: pgmap v158: 460 pgs: 72 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-02-05T07:53:42.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:42.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-13"}]': finished 2024-02-05T07:53:42.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-82038-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:42.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-13", "mode": "writeback"}]: dispatch 2024-02-05T07:53:42.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[29507]: osdmap e132: 8 total, 8 up, 8 in 2024-02-05T07:53:42.945 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-13", "mode": "writeback"}]: dispatch 2024-02-05T07:53:42.945 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2620189832' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi131-82060-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:42 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1530541454' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-82038-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-82038-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:42 smithi157 ceph-mon[33345]: pgmap v158: 460 pgs: 72 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-02-05T07:53:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:42 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-13"}]': finished 2024-02-05T07:53:42.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-82038-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:42.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:42 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-13", "mode": "writeback"}]: dispatch 2024-02-05T07:53:42.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:42 smithi157 ceph-mon[33345]: osdmap e132: 8 total, 8 up, 8 in 2024-02-05T07:53:42.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-13", "mode": "writeback"}]: dispatch 2024-02-05T07:53:42.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:42 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2620189832' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi131-82060-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:43.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:53:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:53:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:53:43.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:43.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:43 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:53:43.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-13", "mode": "writeback"}]': finished 2024-02-05T07:53:43.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2620189832' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi131-82060-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:43.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:43 smithi157 ceph-mon[33345]: osdmap e133: 8 total, 8 up, 8 in 2024-02-05T07:53:44.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:44.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:43 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:53:44.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-13", "mode": "writeback"}]': finished 2024-02-05T07:53:44.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2620189832' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi131-82060-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:44.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:43 smithi131 ceph-mon[29507]: osdmap e133: 8 total, 8 up, 8 in 2024-02-05T07:53:44.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:44.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:43 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:53:44.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-13", "mode": "writeback"}]': finished 2024-02-05T07:53:44.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2620189832' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi131-82060-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:44.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:43 smithi131 ceph-mon[41654]: osdmap e133: 8 total, 8 up, 8 in 2024-02-05T07:53:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: pgmap v161: 428 pgs: 32 creating+peering, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 79 KiB/s wr, 77 op/s 2024-02-05T07:53:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-02-05T07:53:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-02-05T07:53:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-02-05T07:53:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-02-05T07:53:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-02-05T07:53:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-02-05T07:53:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-02-05T07:53:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-02-05T07:53:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-02-05T07:53:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-02-05T07:53:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: osdmap e134: 8 total, 8 up, 8 in 2024-02-05T07:53:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4244182690' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-82038-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:45.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-82038-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:45.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: pgmap v161: 428 pgs: 32 creating+peering, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 79 KiB/s wr, 77 op/s 2024-02-05T07:53:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-02-05T07:53:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-02-05T07:53:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-02-05T07:53:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-02-05T07:53:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-02-05T07:53:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-02-05T07:53:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-02-05T07:53:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-02-05T07:53:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-02-05T07:53:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-02-05T07:53:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: osdmap e134: 8 total, 8 up, 8 in 2024-02-05T07:53:45.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4244182690' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-82038-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:45.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-82038-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:45.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: pgmap v161: 428 pgs: 32 creating+peering, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 79 KiB/s wr, 77 op/s 2024-02-05T07:53:45.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-02-05T07:53:45.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-02-05T07:53:45.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-02-05T07:53:45.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-02-05T07:53:45.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-02-05T07:53:45.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-02-05T07:53:45.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-02-05T07:53:45.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-02-05T07:53:45.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-02-05T07:53:45.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-02-05T07:53:45.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:45.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:53:45.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:45.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: osdmap e134: 8 total, 8 up, 8 in 2024-02-05T07:53:45.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4244182690' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-82038-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:45.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-82038-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:46.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-02-05T07:53:46.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-02-05T07:53:46.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-02-05T07:53:46.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-02-05T07:53:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-02-05T07:53:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-02-05T07:53:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-02-05T07:53:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-02-05T07:53:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-02-05T07:53:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-02-05T07:53:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: 174.8 scrub starts 2024-02-05T07:53:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: 174.8 scrub ok 2024-02-05T07:53:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: 174.7 scrub starts 2024-02-05T07:53:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: 174.7 scrub ok 2024-02-05T07:53:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: 174.9 scrub starts 2024-02-05T07:53:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: 174.9 scrub ok 2024-02-05T07:53:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: 174.3 scrub starts 2024-02-05T07:53:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: 174.3 scrub ok 2024-02-05T07:53:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: 174.6 deep-scrub starts 2024-02-05T07:53:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: 174.6 deep-scrub ok 2024-02-05T07:53:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: 174.2 scrub starts 2024-02-05T07:53:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: 174.2 scrub ok 2024-02-05T07:53:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-82038-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: osdmap e135: 8 total, 8 up, 8 in 2024-02-05T07:53:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/496723064' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-82060-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:46.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-82060-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-02-05T07:53:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-02-05T07:53:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-02-05T07:53:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-02-05T07:53:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-02-05T07:53:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-02-05T07:53:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-02-05T07:53:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-02-05T07:53:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-02-05T07:53:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-02-05T07:53:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: 174.8 scrub starts 2024-02-05T07:53:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: 174.8 scrub ok 2024-02-05T07:53:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: 174.7 scrub starts 2024-02-05T07:53:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: 174.7 scrub ok 2024-02-05T07:53:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: 174.9 scrub starts 2024-02-05T07:53:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: 174.9 scrub ok 2024-02-05T07:53:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: 174.3 scrub starts 2024-02-05T07:53:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: 174.3 scrub ok 2024-02-05T07:53:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: 174.6 deep-scrub starts 2024-02-05T07:53:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: 174.6 deep-scrub ok 2024-02-05T07:53:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: 174.2 scrub starts 2024-02-05T07:53:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: 174.2 scrub ok 2024-02-05T07:53:46.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:46.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-82038-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:46.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: osdmap e135: 8 total, 8 up, 8 in 2024-02-05T07:53:46.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/496723064' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-82060-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:46.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-82060-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-02-05T07:53:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-02-05T07:53:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-02-05T07:53:46.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-02-05T07:53:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-02-05T07:53:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-02-05T07:53:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-02-05T07:53:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-02-05T07:53:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-02-05T07:53:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-02-05T07:53:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: 174.8 scrub starts 2024-02-05T07:53:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: 174.8 scrub ok 2024-02-05T07:53:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: 174.7 scrub starts 2024-02-05T07:53:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: 174.7 scrub ok 2024-02-05T07:53:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: 174.9 scrub starts 2024-02-05T07:53:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: 174.9 scrub ok 2024-02-05T07:53:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: 174.3 scrub starts 2024-02-05T07:53:46.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: 174.3 scrub ok 2024-02-05T07:53:46.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: 174.6 deep-scrub starts 2024-02-05T07:53:46.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: 174.6 deep-scrub ok 2024-02-05T07:53:46.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: 174.2 scrub starts 2024-02-05T07:53:46.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: 174.2 scrub ok 2024-02-05T07:53:46.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:46.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-82038-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:46.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: osdmap e135: 8 total, 8 up, 8 in 2024-02-05T07:53:46.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/496723064' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-82060-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:46.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-82060-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:47.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[41654]: 174.0 scrub starts 2024-02-05T07:53:47.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[41654]: 174.0 scrub ok 2024-02-05T07:53:47.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[41654]: 174.1 scrub starts 2024-02-05T07:53:47.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[41654]: 174.1 scrub ok 2024-02-05T07:53:47.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[41654]: 174.4 scrub starts 2024-02-05T07:53:47.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[41654]: 174.4 scrub ok 2024-02-05T07:53:47.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[41654]: pgmap v164: 492 pgs: 96 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 79 KiB/s wr, 77 op/s 2024-02-05T07:53:47.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:47.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:47.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-82060-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:47.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[41654]: osdmap e136: 8 total, 8 up, 8 in 2024-02-05T07:53:47.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[29507]: 174.0 scrub starts 2024-02-05T07:53:47.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[29507]: 174.0 scrub ok 2024-02-05T07:53:47.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[29507]: 174.1 scrub starts 2024-02-05T07:53:47.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[29507]: 174.1 scrub ok 2024-02-05T07:53:47.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[29507]: 174.4 scrub starts 2024-02-05T07:53:47.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[29507]: 174.4 scrub ok 2024-02-05T07:53:47.171 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[29507]: pgmap v164: 492 pgs: 96 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 79 KiB/s wr, 77 op/s 2024-02-05T07:53:47.171 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:47.171 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:47.171 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-82060-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:47.171 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:46 smithi131 ceph-mon[29507]: osdmap e136: 8 total, 8 up, 8 in 2024-02-05T07:53:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:46 smithi157 ceph-mon[33345]: 174.0 scrub starts 2024-02-05T07:53:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:46 smithi157 ceph-mon[33345]: 174.0 scrub ok 2024-02-05T07:53:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:46 smithi157 ceph-mon[33345]: 174.1 scrub starts 2024-02-05T07:53:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:46 smithi157 ceph-mon[33345]: 174.1 scrub ok 2024-02-05T07:53:47.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:46 smithi157 ceph-mon[33345]: 174.4 scrub starts 2024-02-05T07:53:47.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:46 smithi157 ceph-mon[33345]: 174.4 scrub ok 2024-02-05T07:53:47.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:46 smithi157 ceph-mon[33345]: pgmap v164: 492 pgs: 96 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 79 KiB/s wr, 77 op/s 2024-02-05T07:53:47.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:46 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:47.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:46 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:47.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:46 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-82060-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:47.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:46 smithi157 ceph-mon[33345]: osdmap e136: 8 total, 8 up, 8 in 2024-02-05T07:53:48.459 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:48 smithi157 ceph-mon[33345]: 174.5 scrub starts 2024-02-05T07:53:48.459 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:48 smithi157 ceph-mon[33345]: 174.5 scrub ok 2024-02-05T07:53:48.459 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:48.459 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:48 smithi157 ceph-mon[33345]: osdmap e137: 8 total, 8 up, 8 in 2024-02-05T07:53:48.459 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]: dispatch 2024-02-05T07:53:48.459 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2460539366' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-82038-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:48.459 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-82038-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:48.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:48 smithi131 ceph-mon[41654]: 174.5 scrub starts 2024-02-05T07:53:48.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:48 smithi131 ceph-mon[41654]: 174.5 scrub ok 2024-02-05T07:53:48.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:48.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:48 smithi131 ceph-mon[41654]: osdmap e137: 8 total, 8 up, 8 in 2024-02-05T07:53:48.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]: dispatch 2024-02-05T07:53:48.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2460539366' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-82038-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:48.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-82038-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:48.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:48 smithi131 ceph-mon[29507]: 174.5 scrub starts 2024-02-05T07:53:48.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:48 smithi131 ceph-mon[29507]: 174.5 scrub ok 2024-02-05T07:53:48.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:48.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:48 smithi131 ceph-mon[29507]: osdmap e137: 8 total, 8 up, 8 in 2024-02-05T07:53:48.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]: dispatch 2024-02-05T07:53:48.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2460539366' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-82038-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:48.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-82038-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:48.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:53:48 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:53:49.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:49 smithi157 ceph-mon[33345]: pgmap v167: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:49.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:49.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:49 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:49.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]': finished 2024-02-05T07:53:49.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-82038-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:49.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:49 smithi157 ceph-mon[33345]: osdmap e138: 8 total, 8 up, 8 in 2024-02-05T07:53:49.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]: dispatch 2024-02-05T07:53:49.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3776403676' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-82060-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:49.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-82060-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:49.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[29507]: pgmap v167: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:49.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:49.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:49.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]': finished 2024-02-05T07:53:49.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-82038-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:49.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[29507]: osdmap e138: 8 total, 8 up, 8 in 2024-02-05T07:53:49.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]: dispatch 2024-02-05T07:53:49.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3776403676' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-82060-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:49.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-82060-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:49.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[41654]: pgmap v167: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:49.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:49.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:49.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]': finished 2024-02-05T07:53:49.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-82038-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:49.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[41654]: osdmap e138: 8 total, 8 up, 8 in 2024-02-05T07:53:49.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]: dispatch 2024-02-05T07:53:49.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3776403676' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-82060-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:49.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-82060-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:49.543 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started api_snapshots: Running main() from gmock_main.cc 2024-02-05T07:53:49.543 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-02-05T07:53:49.543 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] Global test environment set-up. 2024-02-05T07:53:49.543 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-02-05T07:53:49.543 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-02-05T07:53:49.543 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (5734 ms) 2024-02-05T07:53:49.544 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-02-05T07:53:49.544 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (3368 ms) 2024-02-05T07:53:49.544 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-02-05T07:53:49.544 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (3504 ms) 2024-02-05T07:53:49.544 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-02-05T07:53:49.544 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (1422 ms) 2024-02-05T07:53:49.544 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (14028 ms total) 2024-02-05T07:53:49.544 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: 2024-02-05T07:53:49.544 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-02-05T07:53:49.544 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-02-05T07:53:49.544 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (5248 ms) 2024-02-05T07:53:49.544 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-02-05T07:53:49.544 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4120 ms) 2024-02-05T07:53:49.544 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-02-05T07:53:49.545 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (4763 ms) 2024-02-05T07:53:49.545 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (14131 ms total) 2024-02-05T07:53:49.545 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: 2024-02-05T07:53:49.545 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-02-05T07:53:49.545 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-02-05T07:53:49.545 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (3004 ms) 2024-02-05T07:53:49.545 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-02-05T07:53:49.545 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (1995 ms) 2024-02-05T07:53:49.545 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-02-05T07:53:49.545 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2015 ms) 2024-02-05T07:53:49.545 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-02-05T07:53:49.545 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (1987 ms) 2024-02-05T07:53:49.545 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (9001 ms total) 2024-02-05T07:53:49.545 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: 2024-02-05T07:53:49.545 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-02-05T07:53:49.545 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-02-05T07:53:49.546 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4032 ms) 2024-02-05T07:53:49.546 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-02-05T07:53:49.546 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (3990 ms) 2024-02-05T07:53:49.546 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8022 ms total) 2024-02-05T07:53:49.546 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: 2024-02-05T07:53:49.546 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] Global test environment tear-down 2024-02-05T07:53:49.546 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (70720 ms total) 2024-02-05T07:53:49.546 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-02-05T07:53:50.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:50.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]': finished 2024-02-05T07:53:50.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-82060-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:50.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:50 smithi157 ceph-mon[33345]: osdmap e139: 8 total, 8 up, 8 in 2024-02-05T07:53:50.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1347935051' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82437-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:50.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82437-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:50.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:50 smithi157 ceph-mon[33345]: pgmap v170: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 54 KiB/s rd, 73 op/s 2024-02-05T07:53:50.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:50.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]': finished 2024-02-05T07:53:50.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-82060-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:50.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:50 smithi131 ceph-mon[41654]: osdmap e139: 8 total, 8 up, 8 in 2024-02-05T07:53:50.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1347935051' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82437-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:50.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82437-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:50.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:50 smithi131 ceph-mon[41654]: pgmap v170: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 54 KiB/s rd, 73 op/s 2024-02-05T07:53:50.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:50.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1832509288' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-82377-15"}]': finished 2024-02-05T07:53:50.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-82060-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:50.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:50 smithi131 ceph-mon[29507]: osdmap e139: 8 total, 8 up, 8 in 2024-02-05T07:53:50.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1347935051' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82437-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:50.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82437-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:50.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:50 smithi131 ceph-mon[29507]: pgmap v170: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 54 KiB/s rd, 73 op/s 2024-02-05T07:53:51.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:51.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82437-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:51.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:51 smithi157 ceph-mon[33345]: osdmap e140: 8 total, 8 up, 8 in 2024-02-05T07:53:51.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1157143616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-82038-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:51.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:51.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82437-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:51.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:51 smithi131 ceph-mon[41654]: osdmap e140: 8 total, 8 up, 8 in 2024-02-05T07:53:51.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1157143616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-82038-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:51.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:51.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82437-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:51.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:51 smithi131 ceph-mon[29507]: osdmap e140: 8 total, 8 up, 8 in 2024-02-05T07:53:51.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1157143616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-82038-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:52.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-05T07:53:52.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:52.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:52 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1157143616' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-82038-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:52 smithi157 ceph-mon[33345]: osdmap e141: 8 total, 8 up, 8 in 2024-02-05T07:53:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app2"}]: dispatch 2024-02-05T07:53:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3377476443' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi131-82060-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:52 smithi157 ceph-mon[33345]: pgmap v173: 484 pgs: 128 unknown, 356 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 56 KiB/s rd, 73 op/s 2024-02-05T07:53:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-05T07:53:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1157143616' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-82038-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[41654]: osdmap e141: 8 total, 8 up, 8 in 2024-02-05T07:53:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app2"}]: dispatch 2024-02-05T07:53:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3377476443' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi131-82060-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:52.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:52.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:52.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[41654]: pgmap v173: 484 pgs: 128 unknown, 356 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 56 KiB/s rd, 73 op/s 2024-02-05T07:53:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-05T07:53:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1157143616' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-82038-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[29507]: osdmap e141: 8 total, 8 up, 8 in 2024-02-05T07:53:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app2"}]: dispatch 2024-02-05T07:53:52.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3377476443' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi131-82060-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:52.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:52.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:52.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-mon[29507]: pgmap v173: 484 pgs: 128 unknown, 356 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 56 KiB/s rd, 73 op/s 2024-02-05T07:53:53.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:53:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:53:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:53:53.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:53.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3377476443' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi131-82060-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:53.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:53.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:53 smithi131 ceph-mon[41654]: osdmap e142: 8 total, 8 up, 8 in 2024-02-05T07:53:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:53:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:53:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:53:53.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:53.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3377476443' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi131-82060-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:53.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:53.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:53 smithi131 ceph-mon[29507]: osdmap e142: 8 total, 8 up, 8 in 2024-02-05T07:53:53.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:53:53.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:53:53.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:53:53.372 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3377476443' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi131-82060-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-82280-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:53 smithi157 ceph-mon[33345]: osdmap e142: 8 total, 8 up, 8 in 2024-02-05T07:53:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:53:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:53:53.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:53:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-05T07:53:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:54 smithi157 ceph-mon[33345]: osdmap e143: 8 total, 8 up, 8 in 2024-02-05T07:53:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3142145723' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi131-82038-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T07:53:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T07:53:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi131-82038-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-82437-16"}]: dispatch 2024-02-05T07:53:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-82437-16"}]: dispatch 2024-02-05T07:53:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-82437-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:54.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:54 smithi157 ceph-mon[33345]: pgmap v176: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:53:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-05T07:53:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[41654]: osdmap e143: 8 total, 8 up, 8 in 2024-02-05T07:53:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3142145723' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi131-82038-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T07:53:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T07:53:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi131-82038-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-82437-16"}]: dispatch 2024-02-05T07:53:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-82437-16"}]: dispatch 2024-02-05T07:53:54.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-82437-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:54.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[41654]: pgmap v176: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:53:54.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:54.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-05T07:53:54.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[29507]: osdmap e143: 8 total, 8 up, 8 in 2024-02-05T07:53:54.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3142145723' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi131-82038-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:54.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T07:53:54.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T07:53:54.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi131-82038-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:54.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-82437-16"}]: dispatch 2024-02-05T07:53:54.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-82437-16"}]: dispatch 2024-02-05T07:53:54.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-82437-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:54.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:54 smithi131 ceph-mon[29507]: pgmap v176: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:53:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-05T07:53:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi131-82038-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-82437-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:55 smithi157 ceph-mon[33345]: osdmap e144: 8 total, 8 up, 8 in 2024-02-05T07:53:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi131-82437-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi131-82437-16"}]: dispatch 2024-02-05T07:53:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T07:53:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T07:53:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/310730714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-82060-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-82060-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:55.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:55.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-05T07:53:55.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi131-82038-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:55.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-82437-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:55.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[29507]: osdmap e144: 8 total, 8 up, 8 in 2024-02-05T07:53:55.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi131-82437-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi131-82437-16"}]: dispatch 2024-02-05T07:53:55.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T07:53:55.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T07:53:55.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/310730714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-82060-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:55.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-82060-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-05T07:53:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi131-82038-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-82437-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:55.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[41654]: osdmap e144: 8 total, 8 up, 8 in 2024-02-05T07:53:55.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi131-82437-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi131-82437-16"}]: dispatch 2024-02-05T07:53:55.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2465004340' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T07:53:55.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T07:53:55.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/310730714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-82060-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:55.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-82060-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1"}]': finished 2024-02-05T07:53:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-82060-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:56 smithi157 ceph-mon[33345]: osdmap e145: 8 total, 8 up, 8 in 2024-02-05T07:53:56.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:56.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1"}]': finished 2024-02-05T07:53:56.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-82060-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:56.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:56 smithi131 ceph-mon[41654]: osdmap e145: 8 total, 8 up, 8 in 2024-02-05T07:53:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:56.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-82280-1","app":"app1","key":"key1"}]': finished 2024-02-05T07:53:56.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-82060-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:56.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:56 smithi131 ceph-mon[29507]: osdmap e145: 8 total, 8 up, 8 in 2024-02-05T07:53:56.526 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-02-05T07:53:56.526 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-02-05T07:53:56.526 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-02-05T07:53:56.526 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-02-05T07:53:56.526 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-02-05T07:53:56.526 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-05T07:53:56.526 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: 2024-02-05T07:53:56.526 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-02-05T07:53:56.526 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: seed 82280 2024-02-05T07:53:56.526 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-02-05T07:53:56.526 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (2 ms) 2024-02-05T07:53:56.526 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-02-05T07:53:56.526 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (259 ms) 2024-02-05T07:53:56.527 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-02-05T07:53:56.527 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (18 ms) 2024-02-05T07:53:56.527 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-02-05T07:53:56.527 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (11 ms) 2024-02-05T07:53:56.527 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-02-05T07:53:56.527 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (8 ms) 2024-02-05T07:53:56.527 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-02-05T07:53:56.527 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (2 ms) 2024-02-05T07:53:56.527 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-02-05T07:53:56.527 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (4 ms) 2024-02-05T07:53:56.527 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-02-05T07:53:56.527 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (6 ms) 2024-02-05T07:53:56.527 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-02-05T07:53:56.527 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (3 ms) 2024-02-05T07:53:56.527 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-02-05T07:53:56.528 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (53 ms) 2024-02-05T07:53:56.528 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-02-05T07:53:56.528 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (8 ms) 2024-02-05T07:53:56.528 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-02-05T07:53:56.528 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (14 ms) 2024-02-05T07:53:56.528 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-02-05T07:53:56.528 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (11 ms) 2024-02-05T07:53:56.528 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-02-05T07:53:56.528 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: osd_max_attr_size = 0 2024-02-05T07:53:56.528 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-02-05T07:53:56.528 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1550 ms) 2024-02-05T07:53:56.528 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-02-05T07:53:56.529 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (346 ms) 2024-02-05T07:53:56.529 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-02-05T07:53:56.529 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-02-05T07:53:56.529 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: done waiting, doing copies 2024-02-05T07:53:56.529 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: waiting for final deep scrubs... 2024-02-05T07:53:56.529 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: done waiting 2024-02-05T07:53:56.529 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60550 ms) 2024-02-05T07:53:56.530 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-02-05T07:53:56.530 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (4 ms) 2024-02-05T07:53:56.530 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-02-05T07:53:56.530 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (3 ms) 2024-02-05T07:53:56.530 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-02-05T07:53:56.530 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4398 ms) 2024-02-05T07:53:56.530 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-02-05T07:53:56.530 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-02-05T07:53:56.530 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-02-05T07:53:56.531 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-02-05T07:53:56.531 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-02-05T07:53:56.531 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-02-05T07:53:57.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:57 smithi157 ceph-mon[33345]: pgmap v179: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:53:57.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:57 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:57.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:57 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi131-82437-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi131-82437-16"}]': finished 2024-02-05T07:53:57.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:57 smithi157 ceph-mon[33345]: osdmap e146: 8 total, 8 up, 8 in 2024-02-05T07:53:57.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:57 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1380353747' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-82038-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:57.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:57 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-82038-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:57.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:57 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:57.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:57 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:57.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:57 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:57.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:57 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:57.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:57 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:57.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:57 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82280-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:57.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:57 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82280-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:57.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[41654]: pgmap v179: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:53:57.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:57.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi131-82437-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi131-82437-16"}]': finished 2024-02-05T07:53:57.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[41654]: osdmap e146: 8 total, 8 up, 8 in 2024-02-05T07:53:57.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1380353747' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-82038-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:57.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-82038-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:57.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:57.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:57.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:57.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:57.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:57.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82280-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:57.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82280-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:57.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[29507]: pgmap v179: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:53:57.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:57.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi131-82437-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi131-82437-16"}]': finished 2024-02-05T07:53:57.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[29507]: osdmap e146: 8 total, 8 up, 8 in 2024-02-05T07:53:57.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1380353747' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-82038-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:57.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-82038-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:57.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:57.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:57.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:53:57.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:57.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:57.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82280-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:57.521 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:57 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82280-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:53:58.465 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:58.466 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-82038-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:58.466 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82280-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:58.466 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:58 smithi157 ceph-mon[33345]: osdmap e147: 8 total, 8 up, 8 in 2024-02-05T07:53:58.466 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2598005577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-82060-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:58.466 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-82060-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:58.466 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82280-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:58.467 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82280-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:58.467 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:58 smithi157 ceph-mon[33345]: pgmap v182: 428 pgs: 72 unknown, 356 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:58.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:58.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-82038-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:58.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82280-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:58.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[41654]: osdmap e147: 8 total, 8 up, 8 in 2024-02-05T07:53:58.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2598005577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-82060-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:58.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-82060-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:58.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82280-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:58.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82280-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:58.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[41654]: pgmap v182: 428 pgs: 72 unknown, 356 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:58.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:58.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-82038-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:58.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82280-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:53:58.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[29507]: osdmap e147: 8 total, 8 up, 8 in 2024-02-05T07:53:58.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2598005577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-82060-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:58.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-82060-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:53:58.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82280-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:58.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82280-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:53:58.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:58 smithi131 ceph-mon[29507]: pgmap v182: 428 pgs: 72 unknown, 356 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:53:58.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:53:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:53:59.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:59.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:59 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:59.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-82060-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:59.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:53:59 smithi157 ceph-mon[33345]: osdmap e148: 8 total, 8 up, 8 in 2024-02-05T07:53:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:59 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-82060-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:53:59 smithi131 ceph-mon[41654]: osdmap e148: 8 total, 8 up, 8 in 2024-02-05T07:53:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:53:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:59 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:59.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-82060-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:53:59.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:53:59 smithi131 ceph-mon[29507]: osdmap e148: 8 total, 8 up, 8 in 2024-02-05T07:54:00.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:00.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:00.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82280-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82280-24"}]': finished 2024-02-05T07:54:00.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:00 smithi157 ceph-mon[33345]: osdmap e149: 8 total, 8 up, 8 in 2024-02-05T07:54:00.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3769556174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-82038-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:00.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-82038-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:00.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:00 smithi157 ceph-mon[33345]: pgmap v185: 404 pgs: 40 unknown, 364 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82280-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82280-24"}]': finished 2024-02-05T07:54:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:00 smithi131 ceph-mon[41654]: osdmap e149: 8 total, 8 up, 8 in 2024-02-05T07:54:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3769556174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-82038-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-82038-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:00 smithi131 ceph-mon[41654]: pgmap v185: 404 pgs: 40 unknown, 364 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:00.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:00.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:00.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82280-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82280-24"}]': finished 2024-02-05T07:54:00.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:00 smithi131 ceph-mon[29507]: osdmap e149: 8 total, 8 up, 8 in 2024-02-05T07:54:00.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3769556174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-82038-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:00.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-82038-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:00.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:00 smithi131 ceph-mon[29507]: pgmap v185: 404 pgs: 40 unknown, 364 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-82038-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:01 smithi157 ceph-mon[33345]: osdmap e150: 8 total, 8 up, 8 in 2024-02-05T07:54:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1200192788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-82060-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:01.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-82060-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:01.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-82038-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:01 smithi131 ceph-mon[41654]: osdmap e150: 8 total, 8 up, 8 in 2024-02-05T07:54:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1200192788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-82060-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-82060-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:01.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:01.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-82038-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:01.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:01 smithi131 ceph-mon[29507]: osdmap e150: 8 total, 8 up, 8 in 2024-02-05T07:54:01.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1200192788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-82060-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:01.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:01.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-82060-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:01.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:01.532 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscP api_snapshots_pp: Running main() from gmock_main.cc 2024-02-05T07:54:01.532 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-02-05T07:54:01.532 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-02-05T07:54:01.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-02-05T07:54:01.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: seed 82437 2024-02-05T07:54:01.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-02-05T07:54:01.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (5734 ms) 2024-02-05T07:54:01.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-02-05T07:54:01.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (3367 ms) 2024-02-05T07:54:01.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-02-05T07:54:01.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (3506 ms) 2024-02-05T07:54:01.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-02-05T07:54:01.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (1420 ms) 2024-02-05T07:54:01.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-02-05T07:54:01.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (6177 ms) 2024-02-05T07:54:01.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (20206 ms total) 2024-02-05T07:54:01.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: 2024-02-05T07:54:01.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-02-05T07:54:01.533 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-02-05T07:54:01.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4158 ms) 2024-02-05T07:54:01.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-02-05T07:54:01.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (3693 ms) 2024-02-05T07:54:01.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-02-05T07:54:01.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (6001 ms) 2024-02-05T07:54:01.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-02-05T07:54:01.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3993 ms) 2024-02-05T07:54:01.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-02-05T07:54:01.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2079 ms) 2024-02-05T07:54:01.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-02-05T07:54:01.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1241-g164d3be4/rpm/el9/BUILD/ceph-19.0.0-1241-g164d3be4/src/test/librados/snapshots_cxx.cc:463: Skipped 2024-02-05T07:54:01.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: 2024-02-05T07:54:01.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-02-05T07:54:01.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-02-05T07:54:01.534 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi131-82437-7 2024-02-05T07:54:01.535 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: waiting for snaps to purge 2024-02-05T07:54:01.535 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17926 ms) 2024-02-05T07:54:01.535 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (37850 ms total) 2024-02-05T07:54:01.535 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: 2024-02-05T07:54:01.535 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-02-05T07:54:01.535 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-02-05T07:54:01.535 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (9 ms) 2024-02-05T07:54:01.535 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-02-05T07:54:01.535 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5056 ms) 2024-02-05T07:54:01.535 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5065 ms total) 2024-02-05T07:54:01.535 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: 2024-02-05T07:54:01.535 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-02-05T07:54:01.535 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-02-05T07:54:01.535 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3021 ms) 2024-02-05T07:54:01.536 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-02-05T07:54:02.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:02.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-82060-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:02.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:02.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:02 smithi157 ceph-mon[33345]: osdmap e151: 8 total, 8 up, 8 in 2024-02-05T07:54:02.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:02.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:02.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:02 smithi157 ceph-mon[33345]: pgmap v188: 436 pgs: 72 unknown, 364 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:54:02.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:02.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-82060-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:02.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:02.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:02 smithi131 ceph-mon[41654]: osdmap e151: 8 total, 8 up, 8 in 2024-02-05T07:54:02.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:02.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:02.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:02 smithi131 ceph-mon[41654]: pgmap v188: 436 pgs: 72 unknown, 364 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:54:02.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:02.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-82060-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:02.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:02.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:02 smithi131 ceph-mon[29507]: osdmap e151: 8 total, 8 up, 8 in 2024-02-05T07:54:02.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:02.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:02.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:02 smithi131 ceph-mon[29507]: pgmap v188: 436 pgs: 72 unknown, 364 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:54:03.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:54:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:54:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:54:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:03 smithi131 ceph-mon[41654]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:03 smithi131 ceph-mon[41654]: osdmap e152: 8 total, 8 up, 8 in 2024-02-05T07:54:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2289807656' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-82038-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:03.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-82038-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:03 smithi131 ceph-mon[29507]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:03 smithi131 ceph-mon[29507]: osdmap e152: 8 total, 8 up, 8 in 2024-02-05T07:54:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2289807656' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-82038-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-82038-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:03 smithi157 ceph-mon[33345]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82280-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:03 smithi157 ceph-mon[33345]: osdmap e152: 8 total, 8 up, 8 in 2024-02-05T07:54:03.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2289807656' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-82038-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:03.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-82038-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:04.085 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:04.085 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-82038-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:04.085 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:04 smithi157 ceph-mon[33345]: osdmap e153: 8 total, 8 up, 8 in 2024-02-05T07:54:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1845555758' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-82060-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-82060-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:04 smithi157 ceph-mon[33345]: pgmap v191: 436 pgs: 32 creating+peering, 32 unknown, 372 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:54:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-82038-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:04 smithi131 ceph-mon[41654]: osdmap e153: 8 total, 8 up, 8 in 2024-02-05T07:54:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1845555758' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-82060-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-82060-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:04 smithi131 ceph-mon[41654]: pgmap v191: 436 pgs: 32 creating+peering, 32 unknown, 372 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:54:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-82038-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:04 smithi131 ceph-mon[29507]: osdmap e153: 8 total, 8 up, 8 in 2024-02-05T07:54:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1845555758' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-82060-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-82060-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:04 smithi131 ceph-mon[29507]: pgmap v191: 436 pgs: 32 creating+peering, 32 unknown, 372 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:54:05.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:05.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-82060-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:05.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:05 smithi157 ceph-mon[33345]: osdmap e154: 8 total, 8 up, 8 in 2024-02-05T07:54:05.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:54:05.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:54:05.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:05.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-82060-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:05.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:05 smithi131 ceph-mon[41654]: osdmap e154: 8 total, 8 up, 8 in 2024-02-05T07:54:05.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:54:05.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:54:05.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:05.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-82060-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:05.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:05 smithi131 ceph-mon[29507]: osdmap e154: 8 total, 8 up, 8 in 2024-02-05T07:54:05.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:54:05.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:54:06.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:06.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82280-24"}]': finished 2024-02-05T07:54:06.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:06 smithi157 ceph-mon[33345]: osdmap e155: 8 total, 8 up, 8 in 2024-02-05T07:54:06.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2226414255' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi131-82038-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:06.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:54:06.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:54:06.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:06 smithi157 ceph-mon[33345]: pgmap v194: 396 pgs: 32 unknown, 364 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:54:06.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:06.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82280-24"}]': finished 2024-02-05T07:54:06.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:06 smithi131 ceph-mon[41654]: osdmap e155: 8 total, 8 up, 8 in 2024-02-05T07:54:06.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2226414255' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi131-82038-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:06.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:54:06.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:54:06.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:06 smithi131 ceph-mon[41654]: pgmap v194: 396 pgs: 32 unknown, 364 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:54:06.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:06.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82280-24"}]': finished 2024-02-05T07:54:06.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:06 smithi131 ceph-mon[29507]: osdmap e155: 8 total, 8 up, 8 in 2024-02-05T07:54:06.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2226414255' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi131-82038-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:06.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/841855493' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:54:06.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82280-24"}]: dispatch 2024-02-05T07:54:06.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:06 smithi131 ceph-mon[29507]: pgmap v194: 396 pgs: 32 unknown, 364 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:54:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:07 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:07 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2226414255' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi131-82038-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82280-24"}]': finished 2024-02-05T07:54:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:07 smithi157 ceph-mon[33345]: osdmap e156: 8 total, 8 up, 8 in 2024-02-05T07:54:07.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:07 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3621196390' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi131-82060-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:07.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:07 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-82437-16"}]: dispatch 2024-02-05T07:54:07.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:07 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:07.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:07 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2226414255' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi131-82038-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:07.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82280-24"}]': finished 2024-02-05T07:54:07.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:07 smithi131 ceph-mon[41654]: osdmap e156: 8 total, 8 up, 8 in 2024-02-05T07:54:07.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:07 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3621196390' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi131-82060-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:07.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:07 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-82437-16"}]: dispatch 2024-02-05T07:54:07.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:07 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:07.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:07 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2226414255' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi131-82038-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:07.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82280-24"}]': finished 2024-02-05T07:54:07.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:07 smithi131 ceph-mon[29507]: osdmap e156: 8 total, 8 up, 8 in 2024-02-05T07:54:07.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:07 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3621196390' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi131-82060-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:07.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:07 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-82437-16"}]: dispatch 2024-02-05T07:54:07.519 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosS snapshots: Running main() from gmock_main.cc 2024-02-05T07:54:07.520 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-02-05T07:54:07.520 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [----------] Global test environment set-up. 2024-02-05T07:54:07.520 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-02-05T07:54:07.520 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-02-05T07:54:07.520 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (14390 ms) 2024-02-05T07:54:07.520 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-02-05T07:54:07.520 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (7463 ms) 2024-02-05T07:54:07.520 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-02-05T07:54:07.520 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (4719 ms) 2024-02-05T07:54:07.520 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-02-05T07:54:07.520 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (5939 ms) 2024-02-05T07:54:07.520 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-02-05T07:54:07.520 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (6852 ms) 2024-02-05T07:54:07.520 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (39363 ms total) 2024-02-05T07:54:07.521 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: 2024-02-05T07:54:07.521 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-02-05T07:54:07.521 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-02-05T07:54:07.521 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (4998 ms) 2024-02-05T07:54:07.521 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-02-05T07:54:07.521 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (6003 ms) 2024-02-05T07:54:07.521 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-02-05T07:54:07.521 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (8258 ms) 2024-02-05T07:54:07.521 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-02-05T07:54:07.521 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (5750 ms) 2024-02-05T07:54:07.521 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-02-05T07:54:07.521 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4003 ms) 2024-02-05T07:54:07.521 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-02-05T07:54:07.521 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi131-83346-11. 2024-02-05T07:54:07.521 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: Waiting for snaps to purge. 2024-02-05T07:54:07.522 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (19999 ms) 2024-02-05T07:54:07.522 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (49011 ms total) 2024-02-05T07:54:07.522 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: 2024-02-05T07:54:07.522 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [----------] Global test environment tear-down 2024-02-05T07:54:07.522 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (88374 ms total) 2024-02-05T07:54:07.522 INFO:tasks.workunit.client.0.smithi131.stdout: snapshots: [ PASSED ] 11 tests. 2024-02-05T07:54:08.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:08.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3621196390' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi131-82060-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:08.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-82437-16"}]': finished 2024-02-05T07:54:08.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:08 smithi157 ceph-mon[33345]: osdmap e157: 8 total, 8 up, 8 in 2024-02-05T07:54:08.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-82437-16"}]: dispatch 2024-02-05T07:54:08.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/316605768' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi131-82280-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:08.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:08 smithi157 ceph-mon[33345]: pgmap v197: 388 pgs: 64 unknown, 324 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:08.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:08.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3621196390' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi131-82060-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:08.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-82437-16"}]': finished 2024-02-05T07:54:08.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:08 smithi131 ceph-mon[41654]: osdmap e157: 8 total, 8 up, 8 in 2024-02-05T07:54:08.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-82437-16"}]: dispatch 2024-02-05T07:54:08.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/316605768' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi131-82280-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:08.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:08 smithi131 ceph-mon[41654]: pgmap v197: 388 pgs: 64 unknown, 324 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:08.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:08.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3621196390' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi131-82060-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:08.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-82437-16"}]': finished 2024-02-05T07:54:08.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:08 smithi131 ceph-mon[29507]: osdmap e157: 8 total, 8 up, 8 in 2024-02-05T07:54:08.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-82437-16"}]: dispatch 2024-02-05T07:54:08.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/316605768' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi131-82280-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:08.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:08 smithi131 ceph-mon[29507]: pgmap v197: 388 pgs: 64 unknown, 324 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:08.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:54:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:54:09.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:09.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:09 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:09.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:09 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:09.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-82437-16"}]': finished 2024-02-05T07:54:09.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/316605768' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi131-82280-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:09.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:09 smithi157 ceph-mon[33345]: osdmap e158: 8 total, 8 up, 8 in 2024-02-05T07:54:09.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/571537936' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi131-82038-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:09.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi131-82038-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:09.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]: dispatch 2024-02-05T07:54:09.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]: dispatch 2024-02-05T07:54:09.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:09 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:54:09.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:09.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:09.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:09.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:09.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-82437-16"}]': finished 2024-02-05T07:54:09.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/316605768' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi131-82280-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:09.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[41654]: osdmap e158: 8 total, 8 up, 8 in 2024-02-05T07:54:09.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/571537936' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi131-82038-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:09.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi131-82038-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:09.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]: dispatch 2024-02-05T07:54:09.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]: dispatch 2024-02-05T07:54:09.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:54:09.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:09.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:09.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:09.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:09.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/134348776' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-82437-16"}]': finished 2024-02-05T07:54:09.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/316605768' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi131-82280-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:09.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[29507]: osdmap e158: 8 total, 8 up, 8 in 2024-02-05T07:54:09.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/571537936' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi131-82038-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:09.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi131-82038-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:09.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]: dispatch 2024-02-05T07:54:09.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]: dispatch 2024-02-05T07:54:09.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:54:09.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:10.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:10.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:10 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:54:10.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:10 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:54:10.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:10 smithi157 ceph-mon[33345]: pgmap v199: 388 pgs: 32 creating+peering, 32 unknown, 324 active+clean; 592 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:54:10.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:10 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:54:10.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi131-82038-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:10.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:10.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:10 smithi157 ceph-mon[33345]: osdmap e159: 8 total, 8 up, 8 in 2024-02-05T07:54:10.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]: dispatch 2024-02-05T07:54:10.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2727037244' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi131-82060-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:54:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:54:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[41654]: pgmap v199: 388 pgs: 32 creating+peering, 32 unknown, 324 active+clean; 592 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:54:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:54:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi131-82038-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[41654]: osdmap e159: 8 total, 8 up, 8 in 2024-02-05T07:54:10.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]: dispatch 2024-02-05T07:54:10.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2727037244' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi131-82060-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:54:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:54:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[29507]: pgmap v199: 388 pgs: 32 creating+peering, 32 unknown, 324 active+clean; 592 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:54:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:54:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi131-82038-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[29507]: osdmap e159: 8 total, 8 up, 8 in 2024-02-05T07:54:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]: dispatch 2024-02-05T07:54:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2727037244' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi131-82060-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:11.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:11 smithi157 ceph-mon[33345]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 3 stray daemon(s) not managed by cephadm) 2024-02-05T07:54:11.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:11.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2727037244' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi131-82060-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:11.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:11 smithi157 ceph-mon[33345]: osdmap e160: 8 total, 8 up, 8 in 2024-02-05T07:54:11.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4101998225' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-82280-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:11.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-82280-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:11.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:11.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:11.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:11.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:11.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-82038-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:11.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-82038-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:11.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[29507]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 3 stray daemon(s) not managed by cephadm) 2024-02-05T07:54:11.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:11.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2727037244' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi131-82060-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:11.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[29507]: osdmap e160: 8 total, 8 up, 8 in 2024-02-05T07:54:11.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4101998225' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-82280-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:11.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-82280-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:11.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:11.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:11.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:11.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:11.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-82038-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:11.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-82038-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:11.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[41654]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 3 stray daemon(s) not managed by cephadm) 2024-02-05T07:54:11.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:11.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2727037244' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi131-82060-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:11.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[41654]: osdmap e160: 8 total, 8 up, 8 in 2024-02-05T07:54:11.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4101998225' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-82280-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:11.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-82280-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:11.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:11.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:11.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:11.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:11.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-82038-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:11.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-82038-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:12.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:12.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:12 smithi157 ceph-mon[33345]: pgmap v202: 388 pgs: 64 unknown, 324 active+clean; 592 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T07:54:12.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]': finished 2024-02-05T07:54:12.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-82280-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:12.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-82038-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:12.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:12 smithi157 ceph-mon[33345]: osdmap e161: 8 total, 8 up, 8 in 2024-02-05T07:54:12.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-82038-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:12.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-82038-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-mon[41654]: pgmap v202: 388 pgs: 64 unknown, 324 active+clean; 592 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T07:54:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]': finished 2024-02-05T07:54:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-82280-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-82038-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-mon[41654]: osdmap e161: 8 total, 8 up, 8 in 2024-02-05T07:54:12.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-82038-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:12.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-82038-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-mon[29507]: pgmap v202: 388 pgs: 64 unknown, 324 active+clean; 592 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T07:54:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]': finished 2024-02-05T07:54:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-82280-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-82038-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-mon[29507]: osdmap e161: 8 total, 8 up, 8 in 2024-02-05T07:54:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-82038-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-82038-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:13.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:54:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:54:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:54:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:13 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:13 smithi131 ceph-mon[29507]: osdmap e162: 8 total, 8 up, 8 in 2024-02-05T07:54:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:13 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2533367983' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-82060-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:13 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-82060-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:13.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:13 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:13.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:13 smithi131 ceph-mon[41654]: osdmap e162: 8 total, 8 up, 8 in 2024-02-05T07:54:13.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:13 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2533367983' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-82060-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:13.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:13 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-82060-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:13.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:13 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:13.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:13 smithi157 ceph-mon[33345]: osdmap e162: 8 total, 8 up, 8 in 2024-02-05T07:54:13.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:13 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2533367983' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-82060-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:13.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:13 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-82060-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:14.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:14.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:14 smithi157 ceph-mon[33345]: pgmap v205: 364 pgs: 32 creating+peering, 332 active+clean; 600 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-05T07:54:14.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-82038-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-82038-27"}]': finished 2024-02-05T07:54:14.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-82060-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:14.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:14 smithi157 ceph-mon[33345]: osdmap e163: 8 total, 8 up, 8 in 2024-02-05T07:54:14.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1150586518' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi131-82280-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:14.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:14.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:14 smithi131 ceph-mon[41654]: pgmap v205: 364 pgs: 32 creating+peering, 332 active+clean; 600 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-05T07:54:14.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-82038-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-82038-27"}]': finished 2024-02-05T07:54:14.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-82060-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:14.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:14 smithi131 ceph-mon[41654]: osdmap e163: 8 total, 8 up, 8 in 2024-02-05T07:54:14.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1150586518' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi131-82280-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:14.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:14.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:14 smithi131 ceph-mon[29507]: pgmap v205: 364 pgs: 32 creating+peering, 332 active+clean; 600 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-05T07:54:14.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-82038-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-82038-27"}]': finished 2024-02-05T07:54:14.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-82060-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:14.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:14 smithi131 ceph-mon[29507]: osdmap e163: 8 total, 8 up, 8 in 2024-02-05T07:54:14.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1150586518' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi131-82280-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:15 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:15 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:54:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1150586518' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi131-82280-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:15 smithi157 ceph-mon[33345]: osdmap e164: 8 total, 8 up, 8 in 2024-02-05T07:54:15.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:15 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:15.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:15.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:15 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:54:15.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:15.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1150586518' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi131-82280-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:15.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:15 smithi131 ceph-mon[41654]: osdmap e164: 8 total, 8 up, 8 in 2024-02-05T07:54:15.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:15 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:15.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:15.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:15 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:54:15.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:15.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1150586518' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi131-82280-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:15.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:15 smithi131 ceph-mon[29507]: osdmap e164: 8 total, 8 up, 8 in 2024-02-05T07:54:16.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:16.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:16 smithi157 ceph-mon[33345]: pgmap v208: 372 pgs: 40 unknown, 332 active+clean; 600 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-05T07:54:16.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:16 smithi157 ceph-mon[33345]: osdmap e165: 8 total, 8 up, 8 in 2024-02-05T07:54:16.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:16.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:16.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2024921663' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi131-82060-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:16.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-82280-36"}]: dispatch 2024-02-05T07:54:16.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-82280-36"}]: dispatch 2024-02-05T07:54:16.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi131-82280-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:16.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:16.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[29507]: pgmap v208: 372 pgs: 40 unknown, 332 active+clean; 600 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-05T07:54:16.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[29507]: osdmap e165: 8 total, 8 up, 8 in 2024-02-05T07:54:16.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:16.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:16.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2024921663' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi131-82060-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:16.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-82280-36"}]: dispatch 2024-02-05T07:54:16.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-82280-36"}]: dispatch 2024-02-05T07:54:16.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi131-82280-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:16.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:16.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[41654]: pgmap v208: 372 pgs: 40 unknown, 332 active+clean; 600 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-05T07:54:16.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[41654]: osdmap e165: 8 total, 8 up, 8 in 2024-02-05T07:54:16.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:16.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:16.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2024921663' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi131-82060-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:16.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-82280-36"}]: dispatch 2024-02-05T07:54:16.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-82280-36"}]: dispatch 2024-02-05T07:54:16.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi131-82280-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:17.419 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:17.419 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-82038-27"}]': finished 2024-02-05T07:54:17.419 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2024921663' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi131-82060-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:17.419 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi131-82280-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:17.419 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:17 smithi131 ceph-mon[41654]: osdmap e166: 8 total, 8 up, 8 in 2024-02-05T07:54:17.420 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi131-82280-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi131-82280-36"}]: dispatch 2024-02-05T07:54:17.420 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:17.420 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:17.420 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:17.420 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-82038-27"}]': finished 2024-02-05T07:54:17.420 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2024921663' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi131-82060-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:17.420 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi131-82280-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:17.420 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:17 smithi131 ceph-mon[29507]: osdmap e166: 8 total, 8 up, 8 in 2024-02-05T07:54:17.420 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi131-82280-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi131-82280-36"}]: dispatch 2024-02-05T07:54:17.420 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:17.420 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-82038-27"}]': finished 2024-02-05T07:54:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2024921663' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi131-82060-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi131-82280-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:17 smithi157 ceph-mon[33345]: osdmap e166: 8 total, 8 up, 8 in 2024-02-05T07:54:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi131-82280-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi131-82280-36"}]: dispatch 2024-02-05T07:54:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2176786344' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-82038-27"}]: dispatch 2024-02-05T07:54:17.578 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: Running main() from gmock_main.cc 2024-02-05T07:54:17.578 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-02-05T07:54:17.578 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [----------] Global test environment set-up. 2024-02-05T07:54:17.579 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-02-05T07:54:17.579 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-02-05T07:54:17.579 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.TooBig (9922 ms) 2024-02-05T07:54:17.579 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-02-05T07:54:17.579 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (7767 ms) 2024-02-05T07:54:17.579 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-02-05T07:54:17.579 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (4592 ms) 2024-02-05T07:54:17.579 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-02-05T07:54:17.579 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3412 ms) 2024-02-05T07:54:17.579 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-02-05T07:54:17.579 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3100 ms) 2024-02-05T07:54:17.579 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-02-05T07:54:17.579 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (3147 ms) 2024-02-05T07:54:17.579 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-02-05T07:54:17.579 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3088 ms) 2024-02-05T07:54:17.580 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-02-05T07:54:17.580 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (2759 ms) 2024-02-05T07:54:17.580 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-02-05T07:54:17.580 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3023 ms) 2024-02-05T07:54:17.580 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-02-05T07:54:17.580 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3238 ms) 2024-02-05T07:54:17.580 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-02-05T07:54:17.580 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (2779 ms) 2024-02-05T07:54:17.580 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-02-05T07:54:17.580 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (2969 ms) 2024-02-05T07:54:17.580 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-02-05T07:54:17.580 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3007 ms) 2024-02-05T07:54:17.580 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-02-05T07:54:17.580 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (3015 ms) 2024-02-05T07:54:17.580 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-02-05T07:54:17.581 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.Flush (3240 ms) 2024-02-05T07:54:17.581 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-02-05T07:54:17.581 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (2768 ms) 2024-02-05T07:54:17.581 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-02-05T07:54:17.581 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (2978 ms) 2024-02-05T07:54:17.581 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-02-05T07:54:17.581 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (2995 ms) 2024-02-05T07:54:17.581 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-02-05T07:54:17.581 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3025 ms) 2024-02-05T07:54:17.581 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-02-05T07:54:17.581 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3018 ms) 2024-02-05T07:54:17.581 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-02-05T07:54:17.581 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (3258 ms) 2024-02-05T07:54:17.581 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-02-05T07:54:17.581 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (2962 ms) 2024-02-05T07:54:17.582 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-02-05T07:54:17.582 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (2769 ms) 2024-02-05T07:54:17.582 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-02-05T07:54:17.582 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3023 ms) 2024-02-05T07:54:17.582 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-02-05T07:54:17.582 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (2956 ms) 2024-02-05T07:54:17.582 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-02-05T07:54:17.582 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3261 ms) 2024-02-05T07:54:17.582 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [----------] 26 tests from LibRadosAio (92071 ms total) 2024-02-05T07:54:17.582 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: 2024-02-05T07:54:17.582 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-02-05T07:54:17.582 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-02-05T07:54:18.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:18.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:18 smithi157 ceph-mon[33345]: pgmap v211: 364 pgs: 32 unknown, 332 active+clean; 600 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:18.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-82038-27"}]': finished 2024-02-05T07:54:18.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:18 smithi157 ceph-mon[33345]: osdmap e167: 8 total, 8 up, 8 in 2024-02-05T07:54:18.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:18.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:18.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:18.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:18.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi131-82038-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:18.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi131-82038-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:18.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:18.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:18.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:18.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[41654]: pgmap v211: 364 pgs: 32 unknown, 332 active+clean; 600 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:18.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-82038-27"}]': finished 2024-02-05T07:54:18.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[41654]: osdmap e167: 8 total, 8 up, 8 in 2024-02-05T07:54:18.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:18.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:18.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:18.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:18.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi131-82038-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:18.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi131-82038-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:18.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:18.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:18.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:18.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[29507]: pgmap v211: 364 pgs: 32 unknown, 332 active+clean; 600 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:18.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-82038-27"}]': finished 2024-02-05T07:54:18.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[29507]: osdmap e167: 8 total, 8 up, 8 in 2024-02-05T07:54:18.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:18.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:18.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:18.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:18.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi131-82038-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:18.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi131-82038-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:18.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:18.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:18.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:54:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:54:19.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:19.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:19 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:19.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi131-82280-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi131-82280-36"}]': finished 2024-02-05T07:54:19.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi131-82038-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:19.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:54:19.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:19 smithi157 ceph-mon[33345]: osdmap e168: 8 total, 8 up, 8 in 2024-02-05T07:54:19.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13"}]: dispatch 2024-02-05T07:54:19.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi131-82038-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:19.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13"}]: dispatch 2024-02-05T07:54:19.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi131-82038-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:19.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/111080772' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi131-82060-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:19.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:19.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:19.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi131-82280-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi131-82280-36"}]': finished 2024-02-05T07:54:19.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi131-82038-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:19.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:54:19.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[29507]: osdmap e168: 8 total, 8 up, 8 in 2024-02-05T07:54:19.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13"}]: dispatch 2024-02-05T07:54:19.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi131-82038-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:19.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13"}]: dispatch 2024-02-05T07:54:19.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi131-82038-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:19.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/111080772' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi131-82060-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:19.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:19.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:19.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi131-82280-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi131-82280-36"}]': finished 2024-02-05T07:54:19.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi131-82038-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:19.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:54:19.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[41654]: osdmap e168: 8 total, 8 up, 8 in 2024-02-05T07:54:19.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13"}]: dispatch 2024-02-05T07:54:19.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi131-82038-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:19.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13"}]: dispatch 2024-02-05T07:54:19.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi131-82038-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:19.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/111080772' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi131-82060-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:20.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:20 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:20.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:20 smithi157 ceph-mon[33345]: pgmap v214: 372 pgs: 40 unknown, 332 active+clean; 609 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-02-05T07:54:20.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:20 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:54:20.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:20 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13"}]': finished 2024-02-05T07:54:20.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:20 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/111080772' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi131-82060-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:20.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:20 smithi157 ceph-mon[33345]: osdmap e169: 8 total, 8 up, 8 in 2024-02-05T07:54:20.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:20 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:20.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:20 smithi131 ceph-mon[41654]: pgmap v214: 372 pgs: 40 unknown, 332 active+clean; 609 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-02-05T07:54:20.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:20 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:54:20.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:20 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13"}]': finished 2024-02-05T07:54:20.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:20 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/111080772' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi131-82060-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:20.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:20 smithi131 ceph-mon[41654]: osdmap e169: 8 total, 8 up, 8 in 2024-02-05T07:54:20.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:20 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:20.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:20 smithi131 ceph-mon[29507]: pgmap v214: 372 pgs: 40 unknown, 332 active+clean; 609 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-02-05T07:54:20.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:20 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:54:20.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:20 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-13"}]': finished 2024-02-05T07:54:20.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:20 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/111080772' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi131-82060-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:20.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:20 smithi131 ceph-mon[29507]: osdmap e169: 8 total, 8 up, 8 in 2024-02-05T07:54:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:21 smithi157 ceph-mon[33345]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi131-82038-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi131-82038-28"}]': finished 2024-02-05T07:54:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:21 smithi157 ceph-mon[33345]: osdmap e170: 8 total, 8 up, 8 in 2024-02-05T07:54:21.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-82280-36"}]: dispatch 2024-02-05T07:54:21.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:21 smithi131 ceph-mon[41654]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:21.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:21.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi131-82038-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi131-82038-28"}]': finished 2024-02-05T07:54:21.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:21 smithi131 ceph-mon[41654]: osdmap e170: 8 total, 8 up, 8 in 2024-02-05T07:54:21.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-82280-36"}]: dispatch 2024-02-05T07:54:21.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:21 smithi131 ceph-mon[29507]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:21.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:21.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi131-82038-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi131-82038-28"}]': finished 2024-02-05T07:54:21.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:21 smithi131 ceph-mon[29507]: osdmap e170: 8 total, 8 up, 8 in 2024-02-05T07:54:21.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-82280-36"}]: dispatch 2024-02-05T07:54:22.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:22.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:22 smithi157 ceph-mon[33345]: pgmap v217: 308 pgs: 8 unknown, 300 active+clean; 609 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-05T07:54:22.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-82280-36"}]': finished 2024-02-05T07:54:22.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:22 smithi157 ceph-mon[33345]: osdmap e171: 8 total, 8 up, 8 in 2024-02-05T07:54:22.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:22.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-82280-36"}]: dispatch 2024-02-05T07:54:22.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:22.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2918473636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi131-82060-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:22.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:22.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-mon[41654]: pgmap v217: 308 pgs: 8 unknown, 300 active+clean; 609 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-05T07:54:22.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-82280-36"}]': finished 2024-02-05T07:54:22.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-mon[41654]: osdmap e171: 8 total, 8 up, 8 in 2024-02-05T07:54:22.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:22.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-82280-36"}]: dispatch 2024-02-05T07:54:22.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:22.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2918473636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi131-82060-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:22.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:22.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-mon[29507]: pgmap v217: 308 pgs: 8 unknown, 300 active+clean; 609 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-05T07:54:22.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-82280-36"}]': finished 2024-02-05T07:54:22.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-mon[29507]: osdmap e171: 8 total, 8 up, 8 in 2024-02-05T07:54:22.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:22.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-82280-36"}]: dispatch 2024-02-05T07:54:22.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:22.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2918473636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi131-82060-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:22.541 INFO:tasks.workunit.client.0.smithi131.stdout: P (67250 ms total) 2024-02-05T07:54:22.541 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: 2024-02-05T07:54:22.541 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-02-05T07:54:22.541 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-02-05T07:54:22.541 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (110 ms) 2024-02-05T07:54:22.541 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (110 ms total) 2024-02-05T07:54:22.541 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: 2024-02-05T07:54:22.541 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-02-05T07:54:22.541 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-02-05T07:54:22.541 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (43 ms) 2024-02-05T07:54:22.541 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-02-05T07:54:22.541 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (2 ms) 2024-02-05T07:54:22.542 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (45 ms total) 2024-02-05T07:54:22.542 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: 2024-02-05T07:54:22.542 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-02-05T07:54:22.542 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-02-05T07:54:22.542 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (40 ms) 2024-02-05T07:54:22.542 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-02-05T07:54:22.542 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (6 ms) 2024-02-05T07:54:22.542 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (46 ms total) 2024-02-05T07:54:22.542 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: 2024-02-05T07:54:22.542 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-02-05T07:54:22.542 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-02-05T07:54:22.542 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (38 ms) 2024-02-05T07:54:22.542 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-02-05T07:54:22.542 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (18 ms) 2024-02-05T07:54:22.542 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (56 ms total) 2024-02-05T07:54:22.542 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: 2024-02-05T07:54:22.543 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-02-05T07:54:22.543 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-02-05T07:54:22.543 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1058 ms) 2024-02-05T07:54:22.543 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1058 ms total) 2024-02-05T07:54:22.543 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: 2024-02-05T07:54:22.543 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-02-05T07:54:22.543 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (103728 ms total) 2024-02-05T07:54:22.543 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-02-05T07:54:23.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:54:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:54:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:54:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-82280-36"}]': finished 2024-02-05T07:54:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2918473636' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi131-82060-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:23 smithi131 ceph-mon[29507]: osdmap e172: 8 total, 8 up, 8 in 2024-02-05T07:54:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:23.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:23.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-82280-36"}]': finished 2024-02-05T07:54:23.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:23.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2918473636' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi131-82060-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:23.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:23 smithi131 ceph-mon[41654]: osdmap e172: 8 total, 8 up, 8 in 2024-02-05T07:54:23.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:23.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:23.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:23.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3888487315' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-82280-36"}]': finished 2024-02-05T07:54:23.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:23.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2918473636' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi131-82060-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:23.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:23 smithi157 ceph-mon[33345]: osdmap e172: 8 total, 8 up, 8 in 2024-02-05T07:54:23.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:23.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:24.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:24 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:24.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:24 smithi157 ceph-mon[33345]: pgmap v220: 364 pgs: 364 active+clean; 609 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:24.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-82038-28"}]': finished 2024-02-05T07:54:24.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:24 smithi157 ceph-mon[33345]: osdmap e173: 8 total, 8 up, 8 in 2024-02-05T07:54:24.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:24 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:24.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:24.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:24 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:24.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:24.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:24 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:24.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:24 smithi131 ceph-mon[41654]: pgmap v220: 364 pgs: 364 active+clean; 609 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:24.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-82038-28"}]': finished 2024-02-05T07:54:24.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:24 smithi131 ceph-mon[41654]: osdmap e173: 8 total, 8 up, 8 in 2024-02-05T07:54:24.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:24 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:24.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:24.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:24 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:24.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:24 smithi131 ceph-mon[29507]: pgmap v220: 364 pgs: 364 active+clean; 609 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-82038-28"}]': finished 2024-02-05T07:54:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:24 smithi131 ceph-mon[29507]: osdmap e173: 8 total, 8 up, 8 in 2024-02-05T07:54:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1079640350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-82038-28"}]: dispatch 2024-02-05T07:54:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:24.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:25.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:25.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:54:25.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:54:25.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-82038-28"}]': finished 2024-02-05T07:54:25.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:54:25.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]': finished 2024-02-05T07:54:25.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: osdmap e174: 8 total, 8 up, 8 in 2024-02-05T07:54:25.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-15"}]: dispatch 2024-02-05T07:54:25.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:25.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-15"}]: dispatch 2024-02-05T07:54:25.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:25.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]: dispatch 2024-02-05T07:54:25.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3577213' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-82060-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:25.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:54:25.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-82060-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:25.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:54:25.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:25.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:25.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:25.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:25.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-82038-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:25.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-82038-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:25.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:25.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:54:25.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:54:25.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-82038-28"}]': finished 2024-02-05T07:54:25.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:54:25.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]': finished 2024-02-05T07:54:25.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: osdmap e174: 8 total, 8 up, 8 in 2024-02-05T07:54:25.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-15"}]: dispatch 2024-02-05T07:54:25.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:25.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-15"}]: dispatch 2024-02-05T07:54:25.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:25.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]: dispatch 2024-02-05T07:54:25.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3577213' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-82060-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:25.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:54:25.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-82060-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:25.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:54:25.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:25.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:25.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:25.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:25.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:25.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:54:25.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:54:25.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-82038-28"}]': finished 2024-02-05T07:54:25.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:54:25.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]': finished 2024-02-05T07:54:25.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: osdmap e174: 8 total, 8 up, 8 in 2024-02-05T07:54:25.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-15"}]: dispatch 2024-02-05T07:54:25.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:25.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-15"}]: dispatch 2024-02-05T07:54:25.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:25.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]: dispatch 2024-02-05T07:54:25.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3577213' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-82060-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:25.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:54:25.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-82060-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:25.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:54:25.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:25.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:25.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:25.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:25.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-82038-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:25.521 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-82038-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:25.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-82038-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:25.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-82038-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:26.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-02-05T07:54:26.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-02-05T07:54:26.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-02-05T07:54:26.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-02-05T07:54:26.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-02-05T07:54:26.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (9951 ms) 2024-02-05T07:54:26.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-02-05T07:54:26.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (19910 ms) 2024-02-05T07:54:26.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-02-05T07:54:26.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6172 ms) 2024-02-05T07:54:26.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-02-05T07:54:26.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (2759 ms) 2024-02-05T07:54:26.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-02-05T07:54:26.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3014 ms) 2024-02-05T07:54:26.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-02-05T07:54:26.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (2986 ms) 2024-02-05T07:54:26.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-02-05T07:54:26.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (3006 ms) 2024-02-05T07:54:26.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-02-05T07:54:26.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (2997 ms) 2024-02-05T07:54:26.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-02-05T07:54:26.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2999 ms) 2024-02-05T07:54:26.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-02-05T07:54:26.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3013 ms) 2024-02-05T07:54:26.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-02-05T07:54:26.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (2999 ms) 2024-02-05T07:54:26.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-02-05T07:54:26.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (2988 ms) 2024-02-05T07:54:26.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-02-05T07:54:26.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3007 ms) 2024-02-05T07:54:26.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-02-05T07:54:26.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3012 ms) 2024-02-05T07:54:26.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-02-05T07:54:26.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3012 ms) 2024-02-05T07:54:26.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-02-05T07:54:26.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3014 ms) 2024-02-05T07:54:26.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-02-05T07:54:26.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (2972 ms) 2024-02-05T07:54:26.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-02-05T07:54:26.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3007 ms) 2024-02-05T07:54:26.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-02-05T07:54:26.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3019 ms) 2024-02-05T07:54:26.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-02-05T07:54:26.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3016 ms) 2024-02-05T07:54:26.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-02-05T07:54:26.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (2965 ms) 2024-02-05T07:54:26.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-02-05T07:54:26.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3004 ms) 2024-02-05T07:54:26.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-02-05T07:54:26.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3010 ms) 2024-02-05T07:54:26.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-02-05T07:54:26.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3295 ms) 2024-02-05T07:54:26.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-02-05T07:54:26.784 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2753 ms) 2024-02-05T07:54:26.784 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-02-05T07:54:26.784 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (2938 ms) 2024-02-05T07:54:26.784 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-02-05T07:54:26.784 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAionapshotsECPP.SnapRemovePP (2005 ms) 2024-02-05T07:54:26.784 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-02-05T07:54:26.784 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2014 ms) 2024-02-05T07:54:26.784 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-02-05T07:54:26.784 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2012 ms) 2024-02-05T07:54:26.785 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (9053 ms total) 2024-02-05T07:54:26.785 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: 2024-02-05T07:54:26.785 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-02-05T07:54:26.785 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-02-05T07:54:26.785 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4039 ms) 2024-02-05T07:54:26.785 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-02-05T07:54:26.785 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (4257 ms) 2024-02-05T07:54:26.785 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-02-05T07:54:26.785 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3697 ms) 2024-02-05T07:54:26.785 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11994 ms total) 2024-02-05T07:54:26.785 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: 2024-02-05T07:54:26.785 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-02-05T07:54:26.785 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (107941 ms total) 2024-02-05T07:54:26.785 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-02-05T07:54:26.785 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-02-05T07:54:26.786 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-02-05T07:54:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: pgmap v223: 356 pgs: 32 unknown, 324 active+clean; 585 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-15"}]': finished 2024-02-05T07:54:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]': finished 2024-02-05T07:54:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-82060-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]': finished 2024-02-05T07:54:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-82038-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-15", "mode": "writeback"}]: dispatch 2024-02-05T07:54:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-82038-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: osdmap e175: 8 total, 8 up, 8 in 2024-02-05T07:54:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]: dispatch 2024-02-05T07:54:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-82038-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-15", "mode": "writeback"}]: dispatch 2024-02-05T07:54:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:54:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:54:27.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:26 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: pgmap v223: 356 pgs: 32 unknown, 324 active+clean; 585 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-15"}]': finished 2024-02-05T07:54:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]': finished 2024-02-05T07:54:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-82060-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]': finished 2024-02-05T07:54:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-82038-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-15", "mode": "writeback"}]: dispatch 2024-02-05T07:54:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-82038-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: osdmap e175: 8 total, 8 up, 8 in 2024-02-05T07:54:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]: dispatch 2024-02-05T07:54:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-82038-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-15", "mode": "writeback"}]: dispatch 2024-02-05T07:54:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:54:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:54:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:27.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: pgmap v223: 356 pgs: 32 unknown, 324 active+clean; 585 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:27.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-15"}]': finished 2024-02-05T07:54:27.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]': finished 2024-02-05T07:54:27.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-82060-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:27.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]': finished 2024-02-05T07:54:27.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-82038-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:27.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-15", "mode": "writeback"}]: dispatch 2024-02-05T07:54:27.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:27.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-82038-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:27.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: osdmap e175: 8 total, 8 up, 8 in 2024-02-05T07:54:27.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]: dispatch 2024-02-05T07:54:27.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-82038-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:27.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-15", "mode": "writeback"}]: dispatch 2024-02-05T07:54:27.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:27.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:54:27.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:54:27.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:26 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:27 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:54:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:27 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]': finished 2024-02-05T07:54:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-15", "mode": "writeback"}]': finished 2024-02-05T07:54:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]': finished 2024-02-05T07:54:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:27 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:27 smithi157 ceph-mon[33345]: osdmap e176: 8 total, 8 up, 8 in 2024-02-05T07:54:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-82124-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-05T07:54:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:27 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-82124-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-05T07:54:28.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-82038-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-82038-29"}]': finished 2024-02-05T07:54:28.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:27 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-82124-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-05T07:54:28.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:27 smithi157 ceph-mon[33345]: osdmap e177: 8 total, 8 up, 8 in 2024-02-05T07:54:28.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:27 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:28.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:27 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/920933513' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi131-82060-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:28.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi131-82060-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:28.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:27 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:28.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:54:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]': finished 2024-02-05T07:54:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-15", "mode": "writeback"}]': finished 2024-02-05T07:54:28.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]': finished 2024-02-05T07:54:28.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:28.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[41654]: osdmap e176: 8 total, 8 up, 8 in 2024-02-05T07:54:28.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-82124-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-05T07:54:28.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-82124-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-05T07:54:28.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-82038-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-82038-29"}]': finished 2024-02-05T07:54:28.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-82124-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-05T07:54:28.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[41654]: osdmap e177: 8 total, 8 up, 8 in 2024-02-05T07:54:28.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:28.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/920933513' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi131-82060-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:28.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi131-82060-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:28.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:28.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:28.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:54:28.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2741776497' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-82437-21"}]': finished 2024-02-05T07:54:28.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-15", "mode": "writeback"}]': finished 2024-02-05T07:54:28.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pg_num","val":"11"}]': finished 2024-02-05T07:54:28.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:28.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[29507]: osdmap e176: 8 total, 8 up, 8 in 2024-02-05T07:54:28.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-82124-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-05T07:54:28.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-82124-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-05T07:54:28.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-82038-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-82038-29"}]': finished 2024-02-05T07:54:28.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-82124-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-05T07:54:28.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[29507]: osdmap e177: 8 total, 8 up, 8 in 2024-02-05T07:54:28.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:28.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/920933513' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi131-82060-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:28.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi131-82060-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:28.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:28.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:28.788 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:54:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:54:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:28 smithi157 ceph-mon[33345]: pgmap v226: 324 pgs: 324 active+clean; 585 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi131-82060-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:54:29.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:29.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:28 smithi157 ceph-mon[33345]: osdmap e178: 8 total, 8 up, 8 in 2024-02-05T07:54:29.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15"}]: dispatch 2024-02-05T07:54:29.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:29.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15"}]: dispatch 2024-02-05T07:54:29.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:54:29.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:54:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[41654]: pgmap v226: 324 pgs: 324 active+clean; 585 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi131-82060-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:54:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:29.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[41654]: osdmap e178: 8 total, 8 up, 8 in 2024-02-05T07:54:29.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15"}]: dispatch 2024-02-05T07:54:29.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:29.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15"}]: dispatch 2024-02-05T07:54:29.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:54:29.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:54:29.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[29507]: pgmap v226: 324 pgs: 324 active+clean; 585 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:29.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi131-82060-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:29.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:54:29.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:29.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[29507]: osdmap e178: 8 total, 8 up, 8 in 2024-02-05T07:54:29.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15"}]: dispatch 2024-02-05T07:54:29.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:29.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15"}]: dispatch 2024-02-05T07:54:29.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:54:29.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:54:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:29 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:29 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:29 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:54:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15"}]': finished 2024-02-05T07:54:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]': finished 2024-02-05T07:54:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:29 smithi157 ceph-mon[33345]: osdmap e179: 8 total, 8 up, 8 in 2024-02-05T07:54:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:54:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15"}]': finished 2024-02-05T07:54:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]': finished 2024-02-05T07:54:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[41654]: osdmap e179: 8 total, 8 up, 8 in 2024-02-05T07:54:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:54:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-15"}]': finished 2024-02-05T07:54:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-82124-1","var":"pgp_num","val":"11"}]': finished 2024-02-05T07:54:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[29507]: osdmap e179: 8 total, 8 up, 8 in 2024-02-05T07:54:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1245317873' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:54:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:30.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:30 smithi157 ceph-mon[33345]: pgmap v229: 364 pgs: 40 creating+peering, 1 peering, 323 active+clean; 585 KiB data, 560 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-05T07:54:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-82038-29"}]': finished 2024-02-05T07:54:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:30 smithi157 ceph-mon[33345]: osdmap e180: 8 total, 8 up, 8 in 2024-02-05T07:54:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4054450992' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi131-82060-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi131-82060-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-82124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-82124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[41654]: pgmap v229: 364 pgs: 40 creating+peering, 1 peering, 323 active+clean; 585 KiB data, 560 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-05T07:54:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-82038-29"}]': finished 2024-02-05T07:54:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[41654]: osdmap e180: 8 total, 8 up, 8 in 2024-02-05T07:54:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4054450992' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi131-82060-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi131-82060-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-82124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-82124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:31.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[29507]: pgmap v229: 364 pgs: 40 creating+peering, 1 peering, 323 active+clean; 585 KiB data, 560 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-05T07:54:31.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-82038-29"}]': finished 2024-02-05T07:54:31.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[29507]: osdmap e180: 8 total, 8 up, 8 in 2024-02-05T07:54:31.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4054450992' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi131-82060-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2949317153' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-82038-29"}]: dispatch 2024-02-05T07:54:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi131-82060-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-82124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-82124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:32 smithi157 ceph-mon[33345]: pgmap v232: 292 pgs: 32 unknown, 260 active+clean; 583 KiB data, 560 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-82038-29"}]': finished 2024-02-05T07:54:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi131-82060-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-82124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:32 smithi157 ceph-mon[33345]: osdmap e181: 8 total, 8 up, 8 in 2024-02-05T07:54:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:32 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:32 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-82124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-82124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:32 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:32 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:32 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-82038-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-82038-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:33.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:54:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:54:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[29507]: pgmap v232: 292 pgs: 32 unknown, 260 active+clean; 583 KiB data, 560 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-82038-29"}]': finished 2024-02-05T07:54:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi131-82060-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-82124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[29507]: osdmap e181: 8 total, 8 up, 8 in 2024-02-05T07:54:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-82124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-82124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-82038-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-82038-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[41654]: pgmap v232: 292 pgs: 32 unknown, 260 active+clean; 583 KiB data, 560 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-82038-29"}]': finished 2024-02-05T07:54:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi131-82060-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:33.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-82124-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:33.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[41654]: osdmap e181: 8 total, 8 up, 8 in 2024-02-05T07:54:33.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:33.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:33.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-82124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:33.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-82124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:33.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:33.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:33.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:33.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:33.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-82038-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:33.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-82038-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:34.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:34.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-82038-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:34.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:33 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-82038-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:34.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:33 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:34.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:33 smithi157 ceph-mon[33345]: osdmap e182: 8 total, 8 up, 8 in 2024-02-05T07:54:34.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-82038-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:34.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:34.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-82124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-82124-2"}]': finished 2024-02-05T07:54:34.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:54:34.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:33 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-17"}]: dispatch 2024-02-05T07:54:34.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:33 smithi157 ceph-mon[33345]: osdmap e183: 8 total, 8 up, 8 in 2024-02-05T07:54:34.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-17"}]: dispatch 2024-02-05T07:54:34.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:34.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-82038-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:34.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-82038-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:34.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:34.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[41654]: osdmap e182: 8 total, 8 up, 8 in 2024-02-05T07:54:34.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-82038-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:34.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:34.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-82124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-82124-2"}]': finished 2024-02-05T07:54:34.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:54:34.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-17"}]: dispatch 2024-02-05T07:54:34.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[41654]: osdmap e183: 8 total, 8 up, 8 in 2024-02-05T07:54:34.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-17"}]: dispatch 2024-02-05T07:54:34.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:34.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-82038-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:34.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-82038-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:34.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:34.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[29507]: osdmap e182: 8 total, 8 up, 8 in 2024-02-05T07:54:34.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-82038-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:34.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:34.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-82124-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-82124-2"}]': finished 2024-02-05T07:54:34.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:54:34.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-17"}]: dispatch 2024-02-05T07:54:34.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[29507]: osdmap e183: 8 total, 8 up, 8 in 2024-02-05T07:54:34.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-17"}]: dispatch 2024-02-05T07:54:34.601 INFO:tasks.workunit.client.0.smithi131.stdout: expected=7 2024-02-05T07:54:34.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:bd63b0f1:::8:head 2024-02-05T07:54:34.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=13:bd63b0f1:::8:head expected=13:bd63b0f1:::8:head 2024-02-05T07:54:34.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:bd63b0f1:::8:head -> 8 2024-02-05T07:54:34.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=8 expected=8 2024-02-05T07:54:34.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:b29083e3:::5:head 2024-02-05T07:54:34.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=13:b29083e3:::5:head expected=13:b29083e3:::5:head 2024-02-05T07:54:34.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:b29083e3:::5:head -> 5 2024-02-05T07:54:34.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=5 expected=5 2024-02-05T07:54:34.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:89d3ae78:::11:head 2024-02-05T07:54:34.602 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=13:89d3ae78:::11:head expected=13:89d3ae78:::11:head 2024-02-05T07:54:34.602 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:89d3ae78:::11:head -> 11 2024-02-05T07:54:34.602 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=11 expected=11 2024-02-05T07:54:34.602 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:863748b0:::15:head 2024-02-05T07:54:34.602 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=13:863748b0:::15:head expected=13:863748b0:::15:head 2024-02-05T07:54:34.602 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:863748b0:::15:head -> 15 2024-02-05T07:54:34.602 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=15 expected=15 2024-02-05T07:54:34.602 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:f905c69b:::2:head 2024-02-05T07:54:34.602 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=13:f905c69b:::2:head expected=13:f905c69b:::2:head 2024-02-05T07:54:34.602 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:f905c69b:::2:head -> 2 2024-02-05T07:54:34.602 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=2 expected=2 2024-02-05T07:54:34.602 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:e960b815:::9:head 2024-02-05T07:54:34.603 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=13:e960b815:::9:head expected=13:e960b815:::9:head 2024-02-05T07:54:34.603 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:e960b815:::9:head -> 9 2024-02-05T07:54:34.603 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=9 expected=9 2024-02-05T07:54:34.603 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:de5d7c5f:::12:head 2024-02-05T07:54:34.603 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=13:de5d7c5f:::12:head expected=13:de5d7c5f:::12:head 2024-02-05T07:54:34.603 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:de5d7c5f:::12:head -> 12 2024-02-05T07:54:34.603 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=12 expected=12 2024-02-05T07:54:34.603 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 13:566253c9:::13:head 2024-02-05T07:54:34.603 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=13:566253c9:::13:head expected=13:566253c9:::13:head 2024-02-05T07:54:34.603 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 13:566253c9:::13:head -> 13 2024-02-05T07:54:34.603 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=13 expected=13 2024-02-05T07:54:34.603 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (186 ms) 2024-02-05T07:54:34.603 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-02-05T07:54:34.604 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (96554 ms) 2024-02-05T07:54:34.604 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-02-05T07:54:34.604 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: split 0/5 -> MIN 13:33333333::::head 2024-02-05T07:54:34.604 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: split 1/5 -> 13:33333333::::head 13:66666666::::head 2024-02-05T07:54:34.604 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: split 2/5 -> 13:66666666::::head 13:99999999::::head 2024-02-05T07:54:34.604 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: split 3/5 -> 13:99999999::::head 13:cccccccc::::head 2024-02-05T07:54:34.604 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: split 4/5 -> 13:cccccccc::::head MAX 2024-02-05T07:54:34.604 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (4020 ms) 2024-02-05T07:54:34.604 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [----------] 7 tests from LibRadosList (102325 ms total) 2024-02-05T07:54:34.604 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 2024-02-05T07:54:34.604 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-02-05T07:54:34.604 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-02-05T07:54:34.604 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1036 ms) 2024-02-05T07:54:34.605 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-02-05T07:54:34.605 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: myset foo1,foo2,foo3 2024-02-05T07:54:34.605 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo1 2024-02-05T07:54:34.605 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo2 2024-02-05T07:54:34.605 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo3 2024-02-05T07:54:34.605 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: myset foo1,foo4,foo5 2024-02-05T07:54:34.605 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo4 2024-02-05T07:54:34.605 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo5 2024-02-05T07:54:34.605 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo1 2024-02-05T07:54:34.605 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: myset foo6,foo7 2024-02-05T07:54:34.605 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo7 2024-02-05T07:54:34.605 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo6 2024-02-05T07:54:34.605 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-05T07:54:34.606 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns1:foo4 2024-02-05T07:54:34.606 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns1:foo5 2024-02-05T07:54:34.606 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns2:foo7 2024-02-05T07:54:34.606 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns2:foo6 2024-02-05T07:54:34.606 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns1:foo1 2024-02-05T07:54:34.606 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: :foo1 2024-02-05T07:54:34.606 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: :foo2 2024-02-05T07:54:34.606 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: :foo3 2024-02-05T07:54:34.606 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (31 ms) 2024-02-05T07:54:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:34 smithi157 ceph-mon[33345]: pgmap v235: 292 pgs: 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3410050530' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-82060-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:34 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-82060-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-82038-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-82038-30"}]': finished 2024-02-05T07:54:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-17"}]': finished 2024-02-05T07:54:35.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-82060-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:35.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:34 smithi157 ceph-mon[33345]: osdmap e184: 8 total, 8 up, 8 in 2024-02-05T07:54:35.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-17", "mode": "writeback"}]: dispatch 2024-02-05T07:54:35.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-17", "mode": "writeback"}]: dispatch 2024-02-05T07:54:35.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[41654]: pgmap v235: 292 pgs: 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:35.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3410050530' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-82060-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:35.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:35.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-82060-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:35.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-82038-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-82038-30"}]': finished 2024-02-05T07:54:35.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-17"}]': finished 2024-02-05T07:54:35.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-82060-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:35.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[41654]: osdmap e184: 8 total, 8 up, 8 in 2024-02-05T07:54:35.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-17", "mode": "writeback"}]: dispatch 2024-02-05T07:54:35.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-17", "mode": "writeback"}]: dispatch 2024-02-05T07:54:35.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[29507]: pgmap v235: 292 pgs: 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:35.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3410050530' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-82060-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:35.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:35.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-82060-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:35.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-82038-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-82038-30"}]': finished 2024-02-05T07:54:35.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-17"}]': finished 2024-02-05T07:54:35.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-82060-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:35.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[29507]: osdmap e184: 8 total, 8 up, 8 in 2024-02-05T07:54:35.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-17", "mode": "writeback"}]: dispatch 2024-02-05T07:54:35.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-17", "mode": "writeback"}]: dispatch 2024-02-05T07:54:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:35 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:54:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-17", "mode": "writeback"}]': finished 2024-02-05T07:54:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:35 smithi157 ceph-mon[33345]: osdmap e185: 8 total, 8 up, 8 in 2024-02-05T07:54:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:36.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:35 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:54:36.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-17", "mode": "writeback"}]': finished 2024-02-05T07:54:36.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:35 smithi131 ceph-mon[41654]: osdmap e185: 8 total, 8 up, 8 in 2024-02-05T07:54:36.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:36.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:36.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:35 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:54:36.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-17", "mode": "writeback"}]': finished 2024-02-05T07:54:36.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:35 smithi131 ceph-mon[29507]: osdmap e185: 8 total, 8 up, 8 in 2024-02-05T07:54:36.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:36.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:36 smithi157 ceph-mon[33345]: pgmap v238: 340 pgs: 48 unknown, 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:54:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-82124-2"}]': finished 2024-02-05T07:54:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:54:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:36 smithi157 ceph-mon[33345]: osdmap e186: 8 total, 8 up, 8 in 2024-02-05T07:54:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17"}]: dispatch 2024-02-05T07:54:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17"}]: dispatch 2024-02-05T07:54:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1629860657' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-82060-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-82060-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:37.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[29507]: pgmap v238: 340 pgs: 48 unknown, 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:54:37.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:37.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:37.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-82124-2"}]': finished 2024-02-05T07:54:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:54:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[29507]: osdmap e186: 8 total, 8 up, 8 in 2024-02-05T07:54:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17"}]: dispatch 2024-02-05T07:54:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17"}]: dispatch 2024-02-05T07:54:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1629860657' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-82060-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-82060-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:37.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[41654]: pgmap v238: 340 pgs: 48 unknown, 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:54:37.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:37.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:37.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-82124-2"}]': finished 2024-02-05T07:54:37.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:54:37.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[41654]: osdmap e186: 8 total, 8 up, 8 in 2024-02-05T07:54:37.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17"}]: dispatch 2024-02-05T07:54:37.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3563091667' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:37.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17"}]: dispatch 2024-02-05T07:54:37.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-82124-2"}]: dispatch 2024-02-05T07:54:37.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:37.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1629860657' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-82060-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:37.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:37.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-82060-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:38.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:37 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:54:38.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17"}]': finished 2024-02-05T07:54:38.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-82124-2"}]': finished 2024-02-05T07:54:38.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-82038-30"}]': finished 2024-02-05T07:54:38.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-82060-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:38.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:38.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:37 smithi157 ceph-mon[33345]: osdmap e187: 8 total, 8 up, 8 in 2024-02-05T07:54:38.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:38.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:37 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:54:38.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17"}]': finished 2024-02-05T07:54:38.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-82124-2"}]': finished 2024-02-05T07:54:38.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-82038-30"}]': finished 2024-02-05T07:54:38.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-82060-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:38.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:38.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:37 smithi131 ceph-mon[29507]: osdmap e187: 8 total, 8 up, 8 in 2024-02-05T07:54:38.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:38.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:37 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:54:38.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-17"}]': finished 2024-02-05T07:54:38.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-82124-2"}]': finished 2024-02-05T07:54:38.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-82038-30"}]': finished 2024-02-05T07:54:38.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-82060-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:38.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2500500023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:38.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:37 smithi131 ceph-mon[41654]: osdmap e187: 8 total, 8 up, 8 in 2024-02-05T07:54:38.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-82038-30"}]: dispatch 2024-02-05T07:54:38.831 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:54:39.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: pgmap v241: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1629860657' entity='client.admin' cmd=[{ 2024-02-05T07:54:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: "prefix": "osd pool set", 2024-02-05T07:54:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: "pool": "PoolEIOFlag_smithi131-82060-33", 2024-02-05T07:54:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: "var": "eio", 2024-02-05T07:54:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: "val": "true" 2024-02-05T07:54:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: }]: dispatch 2024-02-05T07:54:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{ 2024-02-05T07:54:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: "prefix": "osd pool set", 2024-02-05T07:54:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: "pool": "PoolEIOFlag_smithi131-82060-33", 2024-02-05T07:54:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: "var": "eio", 2024-02-05T07:54:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: "val": "true" 2024-02-05T07:54:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: }]: dispatch 2024-02-05T07:54:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-82038-30"}]': finished 2024-02-05T07:54:39.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{ 2024-02-05T07:54:39.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: "prefix": "osd pool set", 2024-02-05T07:54:39.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: "pool": "PoolEIOFlag_smithi131-82060-33", 2024-02-05T07:54:39.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: "var": "eio", 2024-02-05T07:54:39.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: "val": "true" 2024-02-05T07:54:39.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: }]': finished 2024-02-05T07:54:39.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: osdmap e188: 8 total, 8 up, 8 in 2024-02-05T07:54:39.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1943847273' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82124-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:39.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82124-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:39.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:39.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:39.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:39.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:39.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi131-82038-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:39.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi131-82038-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:39.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: pgmap v241: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:39.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1629860657' entity='client.admin' cmd=[{ 2024-02-05T07:54:39.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: "prefix": "osd pool set", 2024-02-05T07:54:39.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: "pool": "PoolEIOFlag_smithi131-82060-33", 2024-02-05T07:54:39.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: "var": "eio", 2024-02-05T07:54:39.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: "val": "true" 2024-02-05T07:54:39.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: }]: dispatch 2024-02-05T07:54:39.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{ 2024-02-05T07:54:39.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: "prefix": "osd pool set", 2024-02-05T07:54:39.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: "pool": "PoolEIOFlag_smithi131-82060-33", 2024-02-05T07:54:39.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: "var": "eio", 2024-02-05T07:54:39.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: "val": "true" 2024-02-05T07:54:39.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: }]: dispatch 2024-02-05T07:54:39.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-82038-30"}]': finished 2024-02-05T07:54:39.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{ 2024-02-05T07:54:39.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: "prefix": "osd pool set", 2024-02-05T07:54:39.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: "pool": "PoolEIOFlag_smithi131-82060-33", 2024-02-05T07:54:39.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: "var": "eio", 2024-02-05T07:54:39.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: "val": "true" 2024-02-05T07:54:39.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: }]': finished 2024-02-05T07:54:39.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: osdmap e188: 8 total, 8 up, 8 in 2024-02-05T07:54:39.273 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1943847273' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82124-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:39.273 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82124-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:39.273 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:39.273 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:39.273 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:39.273 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:39.274 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi131-82038-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:39.274 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi131-82038-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:39.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: pgmap v241: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:39.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1629860657' entity='client.admin' cmd=[{ 2024-02-05T07:54:39.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: "prefix": "osd pool set", 2024-02-05T07:54:39.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: "pool": "PoolEIOFlag_smithi131-82060-33", 2024-02-05T07:54:39.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: "var": "eio", 2024-02-05T07:54:39.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: "val": "true" 2024-02-05T07:54:39.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: }]: dispatch 2024-02-05T07:54:39.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{ 2024-02-05T07:54:39.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: "prefix": "osd pool set", 2024-02-05T07:54:39.275 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: "pool": "PoolEIOFlag_smithi131-82060-33", 2024-02-05T07:54:39.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: "var": "eio", 2024-02-05T07:54:39.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: "val": "true" 2024-02-05T07:54:39.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: }]: dispatch 2024-02-05T07:54:39.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-82038-30"}]': finished 2024-02-05T07:54:39.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{ 2024-02-05T07:54:39.276 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: "prefix": "osd pool set", 2024-02-05T07:54:39.277 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: "pool": "PoolEIOFlag_smithi131-82060-33", 2024-02-05T07:54:39.277 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: "var": "eio", 2024-02-05T07:54:39.277 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: "val": "true" 2024-02-05T07:54:39.277 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: }]': finished 2024-02-05T07:54:39.277 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: osdmap e188: 8 total, 8 up, 8 in 2024-02-05T07:54:39.277 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1943847273' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82124-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:39.277 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82124-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:39.278 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:39.278 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:39.278 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:39.278 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:39.278 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi131-82038-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:39.278 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi131-82038-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:39 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82124-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi131-82038-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:39 smithi157 ceph-mon[33345]: osdmap e189: 8 total, 8 up, 8 in 2024-02-05T07:54:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:39 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi131-82038-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:39 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi131-82038-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:39 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1943847273' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-82124-3","pool2":"test-rados-api-smithi131-82124-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-05T07:54:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-82124-3","pool2":"test-rados-api-smithi131-82124-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-05T07:54:40.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:40.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82124-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:40.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi131-82038-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:40.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[29507]: osdmap e189: 8 total, 8 up, 8 in 2024-02-05T07:54:40.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi131-82038-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:40.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi131-82038-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1943847273' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-82124-3","pool2":"test-rados-api-smithi131-82124-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-05T07:54:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-82124-3","pool2":"test-rados-api-smithi131-82124-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-05T07:54:40.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:40.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82124-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:40.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi131-82038-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:40.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[41654]: osdmap e189: 8 total, 8 up, 8 in 2024-02-05T07:54:40.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi131-82038-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:40.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:40.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi131-82038-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:40.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:40.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1943847273' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-82124-3","pool2":"test-rados-api-smithi131-82124-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-05T07:54:40.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-82124-3","pool2":"test-rados-api-smithi131-82124-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-05T07:54:40.551 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-02-05T07:54:40.552 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 1 0 2024-02-05T07:54:40.552 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 10 0 2024-02-05T07:54:40.552 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 13 0 2024-02-05T07:54:40.552 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 7 0 2024-02-05T07:54:40.552 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 14 0 2024-02-05T07:54:40.552 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 0 0 2024-02-05T07:54:40.552 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 15 0 2024-02-05T07:54:40.552 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 11 0 2024-02-05T07:54:40.552 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 5 0 2024-02-05T07:54:40.552 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 8 0 2024-02-05T07:54:40.552 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 6 0 2024-02-05T07:54:40.552 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 3 0 2024-02-05T07:54:40.552 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 4 0 2024-02-05T07:54:40.552 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 12 0 2024-02-05T07:54:40.553 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 9 0 2024-02-05T07:54:40.553 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 2 0 2024-02-05T07:54:40.553 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-05T07:54:40.553 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (45 ms) 2024-02-05T07:54:40.553 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [----------] 3 tests from LibRadosListEC (1112 ms total) 2024-02-05T07:54:40.553 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 2024-02-05T07:54:40.553 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-02-05T07:54:40.553 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-02-05T07:54:40.553 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3003 ms) 2024-02-05T07:54:40.553 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [----------] 1 test from LibRadosListNP (3003 ms total) 2024-02-05T07:54:40.553 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 2024-02-05T07:54:40.553 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [----------] Global test environment tear-down 2024-02-05T07:54:40.553 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [==========] 11 tests from 3 test suites ran. (121796 ms total) 2024-02-05T07:54:40.553 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ PASSED ] 11 tests. 2024-02-05T07:54:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:40 smithi157 ceph-mon[33345]: pgmap v244: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-02-05T07:54:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-82124-3","pool2":"test-rados-api-smithi131-82124-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-05T07:54:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:40 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:40 smithi157 ceph-mon[33345]: osdmap e190: 8 total, 8 up, 8 in 2024-02-05T07:54:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:40 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1348639476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-82060-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-82060-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:41.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:40 smithi131 ceph-mon[29507]: pgmap v244: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-02-05T07:54:41.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:41.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-82124-3","pool2":"test-rados-api-smithi131-82124-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-05T07:54:41.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:40 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:41.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:40 smithi131 ceph-mon[29507]: osdmap e190: 8 total, 8 up, 8 in 2024-02-05T07:54:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:40 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1348639476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-82060-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-82060-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:41.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:40 smithi131 ceph-mon[41654]: pgmap v244: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-02-05T07:54:41.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:41.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-82124-3","pool2":"test-rados-api-smithi131-82124-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-05T07:54:41.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:40 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:41.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:40 smithi131 ceph-mon[41654]: osdmap e190: 8 total, 8 up, 8 in 2024-02-05T07:54:41.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:40 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1348639476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-82060-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:41.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:41.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-82060-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:41 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi131-82038-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi131-82038-31"}]': finished 2024-02-05T07:54:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:54:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-82060-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:42.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:41 smithi157 ceph-mon[33345]: osdmap e191: 8 total, 8 up, 8 in 2024-02-05T07:54:42.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-19"}]: dispatch 2024-02-05T07:54:42.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-19"}]: dispatch 2024-02-05T07:54:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:41 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi131-82038-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi131-82038-31"}]': finished 2024-02-05T07:54:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:54:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-82060-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:41 smithi131 ceph-mon[29507]: osdmap e191: 8 total, 8 up, 8 in 2024-02-05T07:54:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-19"}]: dispatch 2024-02-05T07:54:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-19"}]: dispatch 2024-02-05T07:54:42.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:41 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:42.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi131-82038-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi131-82038-31"}]': finished 2024-02-05T07:54:42.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:54:42.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-82060-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:42.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:41 smithi131 ceph-mon[41654]: osdmap e191: 8 total, 8 up, 8 in 2024-02-05T07:54:42.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-19"}]: dispatch 2024-02-05T07:54:42.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-19"}]: dispatch 2024-02-05T07:54:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:42 smithi157 ceph-mon[33345]: pgmap v247: 324 pgs: 64 unknown, 260 active+clean; 583 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:54:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-19"}]': finished 2024-02-05T07:54:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:42 smithi157 ceph-mon[33345]: osdmap e192: 8 total, 8 up, 8 in 2024-02-05T07:54:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:42 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-19", "mode": "writeback"}]: dispatch 2024-02-05T07:54:43.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-19", "mode": "writeback"}]: dispatch 2024-02-05T07:54:43.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:54:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:54:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:54:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:42 smithi131 ceph-mon[29507]: pgmap v247: 324 pgs: 64 unknown, 260 active+clean; 583 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:54:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-19"}]': finished 2024-02-05T07:54:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:42 smithi131 ceph-mon[29507]: osdmap e192: 8 total, 8 up, 8 in 2024-02-05T07:54:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:42 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-19", "mode": "writeback"}]: dispatch 2024-02-05T07:54:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-19", "mode": "writeback"}]: dispatch 2024-02-05T07:54:43.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:42 smithi131 ceph-mon[41654]: pgmap v247: 324 pgs: 64 unknown, 260 active+clean; 583 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:54:43.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-19"}]': finished 2024-02-05T07:54:43.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:42 smithi131 ceph-mon[41654]: osdmap e192: 8 total, 8 up, 8 in 2024-02-05T07:54:43.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:42 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-19", "mode": "writeback"}]: dispatch 2024-02-05T07:54:43.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-19", "mode": "writeback"}]: dispatch 2024-02-05T07:54:44.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:43 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:54:44.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-19", "mode": "writeback"}]': finished 2024-02-05T07:54:44.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:43 smithi157 ceph-mon[33345]: osdmap e193: 8 total, 8 up, 8 in 2024-02-05T07:54:44.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1883308295' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi131-82060-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:44.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:44.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:44.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:44.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:44.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:43 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:54:44.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-19", "mode": "writeback"}]': finished 2024-02-05T07:54:44.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:43 smithi131 ceph-mon[41654]: osdmap e193: 8 total, 8 up, 8 in 2024-02-05T07:54:44.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1883308295' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi131-82060-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:44.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:44.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:44.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:44.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:44.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:43 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:54:44.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-19", "mode": "writeback"}]': finished 2024-02-05T07:54:44.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:43 smithi131 ceph-mon[29507]: osdmap e193: 8 total, 8 up, 8 in 2024-02-05T07:54:44.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1883308295' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi131-82060-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:44.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:44.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:44.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:44.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:44 smithi157 ceph-mon[33345]: pgmap v250: 300 pgs: 300 active+clean; 595 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1 op/s 2024-02-05T07:54:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:44 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1883308295' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi131-82060-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-82038-31"}]': finished 2024-02-05T07:54:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:54:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19"}]: dispatch 2024-02-05T07:54:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:44 smithi157 ceph-mon[33345]: osdmap e194: 8 total, 8 up, 8 in 2024-02-05T07:54:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19"}]: dispatch 2024-02-05T07:54:45.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[41654]: pgmap v250: 300 pgs: 300 active+clean; 595 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1 op/s 2024-02-05T07:54:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1883308295' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi131-82060-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-82038-31"}]': finished 2024-02-05T07:54:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:54:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19"}]: dispatch 2024-02-05T07:54:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[41654]: osdmap e194: 8 total, 8 up, 8 in 2024-02-05T07:54:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19"}]: dispatch 2024-02-05T07:54:45.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[29507]: pgmap v250: 300 pgs: 300 active+clean; 595 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1 op/s 2024-02-05T07:54:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1883308295' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi131-82060-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-82038-31"}]': finished 2024-02-05T07:54:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:54:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19"}]: dispatch 2024-02-05T07:54:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[29507]: osdmap e194: 8 total, 8 up, 8 in 2024-02-05T07:54:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/87832422' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19"}]: dispatch 2024-02-05T07:54:45.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-82038-31"}]: dispatch 2024-02-05T07:54:46.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:45 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:54:46.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19"}]': finished 2024-02-05T07:54:46.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-82038-31"}]': finished 2024-02-05T07:54:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:45 smithi157 ceph-mon[33345]: osdmap e195: 8 total, 8 up, 8 in 2024-02-05T07:54:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-82038-32"}]: dispatch 2024-02-05T07:54:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-82038-32"}]: dispatch 2024-02-05T07:54:46.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi131-82038-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:46.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:45 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:54:46.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19"}]': finished 2024-02-05T07:54:46.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-82038-31"}]': finished 2024-02-05T07:54:46.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:45 smithi131 ceph-mon[29507]: osdmap e195: 8 total, 8 up, 8 in 2024-02-05T07:54:46.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-82038-32"}]: dispatch 2024-02-05T07:54:46.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-82038-32"}]: dispatch 2024-02-05T07:54:46.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi131-82038-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:46.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:45 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:54:46.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-19"}]': finished 2024-02-05T07:54:46.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-82038-31"}]': finished 2024-02-05T07:54:46.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:45 smithi131 ceph-mon[41654]: osdmap e195: 8 total, 8 up, 8 in 2024-02-05T07:54:46.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-82038-32"}]: dispatch 2024-02-05T07:54:46.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-82038-32"}]: dispatch 2024-02-05T07:54:46.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi131-82038-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:47.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:46 smithi131 ceph-mon[41654]: pgmap v253: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:47.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:46 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:47.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:46 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi131-82038-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:47.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:46 smithi131 ceph-mon[41654]: osdmap e196: 8 total, 8 up, 8 in 2024-02-05T07:54:47.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:46 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi131-82038-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi131-82038-32"}]: dispatch 2024-02-05T07:54:47.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:46 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3892774834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:47.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:46 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:47.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:46 smithi131 ceph-mon[29507]: pgmap v253: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:47.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:46 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:47.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:46 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi131-82038-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:47.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:46 smithi131 ceph-mon[29507]: osdmap e196: 8 total, 8 up, 8 in 2024-02-05T07:54:47.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:46 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi131-82038-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi131-82038-32"}]: dispatch 2024-02-05T07:54:47.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:46 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3892774834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:47.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:46 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:46 smithi157 ceph-mon[33345]: pgmap v253: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:46 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:46 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi131-82038-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:46 smithi157 ceph-mon[33345]: osdmap e196: 8 total, 8 up, 8 in 2024-02-05T07:54:47.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:46 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi131-82038-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi131-82038-32"}]: dispatch 2024-02-05T07:54:47.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:46 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3892774834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:47.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:46 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:48.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:54:48 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:54:48.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:48 smithi157 ceph-mon[33345]: pgmap v256: 292 pgs: 32 unknown, 260 active+clean; 583 KiB data, 574 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:48.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:48.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:48 smithi157 ceph-mon[33345]: osdmap e197: 8 total, 8 up, 8 in 2024-02-05T07:54:48.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:48.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:49.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:48 smithi131 ceph-mon[41654]: pgmap v256: 292 pgs: 32 unknown, 260 active+clean; 583 KiB data, 574 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:49.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:49.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:48 smithi131 ceph-mon[41654]: osdmap e197: 8 total, 8 up, 8 in 2024-02-05T07:54:49.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:49.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:49.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:48 smithi131 ceph-mon[29507]: pgmap v256: 292 pgs: 32 unknown, 260 active+clean; 583 KiB data, 574 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:49.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:49.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:48 smithi131 ceph-mon[29507]: osdmap e197: 8 total, 8 up, 8 in 2024-02-05T07:54:49.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:49.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:49 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi131-82038-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi131-82038-32"}]': finished 2024-02-05T07:54:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:49 smithi157 ceph-mon[33345]: osdmap e198: 8 total, 8 up, 8 in 2024-02-05T07:54:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1723475742' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:49 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi131-82038-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi131-82038-32"}]': finished 2024-02-05T07:54:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:49 smithi131 ceph-mon[41654]: osdmap e198: 8 total, 8 up, 8 in 2024-02-05T07:54:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1723475742' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:50.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:50.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:49 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:50.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi131-82038-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi131-82038-32"}]': finished 2024-02-05T07:54:50.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:50.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:49 smithi131 ceph-mon[29507]: osdmap e198: 8 total, 8 up, 8 in 2024-02-05T07:54:50.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1723475742' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:50.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:50.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:50.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:50 smithi157 ceph-mon[33345]: pgmap v259: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:54:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:54:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:50 smithi157 ceph-mon[33345]: osdmap e199: 8 total, 8 up, 8 in 2024-02-05T07:54:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-21"}]: dispatch 2024-02-05T07:54:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-21"}]: dispatch 2024-02-05T07:54:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:50 smithi131 ceph-mon[41654]: pgmap v259: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:54:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:54:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:50 smithi131 ceph-mon[41654]: osdmap e199: 8 total, 8 up, 8 in 2024-02-05T07:54:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-21"}]: dispatch 2024-02-05T07:54:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-21"}]: dispatch 2024-02-05T07:54:51.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:50 smithi131 ceph-mon[29507]: pgmap v259: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:54:51.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:51.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:54:51.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:50 smithi131 ceph-mon[29507]: osdmap e199: 8 total, 8 up, 8 in 2024-02-05T07:54:51.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-21"}]: dispatch 2024-02-05T07:54:51.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-21"}]: dispatch 2024-02-05T07:54:51.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-21"}]': finished 2024-02-05T07:54:51.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:51 smithi157 ceph-mon[33345]: osdmap e200: 8 total, 8 up, 8 in 2024-02-05T07:54:51.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-21", "mode": "writeback"}]: dispatch 2024-02-05T07:54:51.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-82038-32"}]: dispatch 2024-02-05T07:54:51.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-21", "mode": "writeback"}]: dispatch 2024-02-05T07:54:51.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3090819894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:51.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:51.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:51 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:51.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:51 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:54:51.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-82038-32"}]': finished 2024-02-05T07:54:51.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-21", "mode": "writeback"}]': finished 2024-02-05T07:54:51.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:51.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:51 smithi157 ceph-mon[33345]: osdmap e201: 8 total, 8 up, 8 in 2024-02-05T07:54:51.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-82038-32"}]: dispatch 2024-02-05T07:54:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-21"}]': finished 2024-02-05T07:54:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[41654]: osdmap e200: 8 total, 8 up, 8 in 2024-02-05T07:54:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-21", "mode": "writeback"}]: dispatch 2024-02-05T07:54:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-82038-32"}]: dispatch 2024-02-05T07:54:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-21", "mode": "writeback"}]: dispatch 2024-02-05T07:54:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3090819894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:52.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:52.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:52.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:54:52.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-82038-32"}]': finished 2024-02-05T07:54:52.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-21", "mode": "writeback"}]': finished 2024-02-05T07:54:52.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:52.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[41654]: osdmap e201: 8 total, 8 up, 8 in 2024-02-05T07:54:52.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-82038-32"}]: dispatch 2024-02-05T07:54:52.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-21"}]': finished 2024-02-05T07:54:52.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[29507]: osdmap e200: 8 total, 8 up, 8 in 2024-02-05T07:54:52.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-21", "mode": "writeback"}]: dispatch 2024-02-05T07:54:52.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-82038-32"}]: dispatch 2024-02-05T07:54:52.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-21", "mode": "writeback"}]: dispatch 2024-02-05T07:54:52.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3090819894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:52.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:52.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:52.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:54:52.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-82038-32"}]': finished 2024-02-05T07:54:52.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-21", "mode": "writeback"}]': finished 2024-02-05T07:54:52.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-82060-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:52.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[29507]: osdmap e201: 8 total, 8 up, 8 in 2024-02-05T07:54:52.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-82038-32"}]: dispatch 2024-02-05T07:54:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-mon[41654]: pgmap v262: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:54:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-82038-32"}]': finished 2024-02-05T07:54:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:54:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21"}]: dispatch 2024-02-05T07:54:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-mon[41654]: osdmap e202: 8 total, 8 up, 8 in 2024-02-05T07:54:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21"}]: dispatch 2024-02-05T07:54:52.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-mon[29507]: pgmap v262: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:54:52.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:52.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:52.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-82038-32"}]': finished 2024-02-05T07:54:52.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:54:52.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21"}]: dispatch 2024-02-05T07:54:52.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-mon[29507]: osdmap e202: 8 total, 8 up, 8 in 2024-02-05T07:54:52.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21"}]: dispatch 2024-02-05T07:54:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:52 smithi157 ceph-mon[33345]: pgmap v262: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:54:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:54:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1264689315' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-82038-32"}]': finished 2024-02-05T07:54:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:54:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21"}]: dispatch 2024-02-05T07:54:52.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:52 smithi157 ceph-mon[33345]: osdmap e202: 8 total, 8 up, 8 in 2024-02-05T07:54:52.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21"}]: dispatch 2024-02-05T07:54:53.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:54:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:54:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:54:53.872 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:53.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:53.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:53.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:53.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi131-82038-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:53.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi131-82038-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:53.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:53 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:54:53.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21"}]': finished 2024-02-05T07:54:53.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi131-82038-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:53.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:53 smithi157 ceph-mon[33345]: osdmap e203: 8 total, 8 up, 8 in 2024-02-05T07:54:53.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi131-82038-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:53.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi131-82038-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi131-82038-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi131-82038-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:54:54.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21"}]': finished 2024-02-05T07:54:54.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi131-82038-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:54.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[41654]: osdmap e203: 8 total, 8 up, 8 in 2024-02-05T07:54:54.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi131-82038-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:54.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi131-82038-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:54.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:54.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:54.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:54.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:54.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi131-82038-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:54.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi131-82038-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:54:54.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:54:54.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-21"}]': finished 2024-02-05T07:54:54.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi131-82038-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:54:54.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[29507]: osdmap e203: 8 total, 8 up, 8 in 2024-02-05T07:54:54.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi131-82038-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:54.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi131-82038-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:55.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:54 smithi157 ceph-mon[33345]: pgmap v265: 356 pgs: 356 active+clean; 584 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T07:54:55.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:54 smithi131 ceph-mon[41654]: pgmap v265: 356 pgs: 356 active+clean; 584 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T07:54:55.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:54 smithi131 ceph-mon[29507]: pgmap v265: 356 pgs: 356 active+clean; 584 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T07:54:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:55 smithi157 ceph-mon[33345]: osdmap e204: 8 total, 8 up, 8 in 2024-02-05T07:54:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi131-82038-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi131-82038-33"}]': finished 2024-02-05T07:54:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:55 smithi157 ceph-mon[33345]: osdmap e205: 8 total, 8 up, 8 in 2024-02-05T07:54:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1718759709' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:56.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:56.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:55 smithi131 ceph-mon[41654]: osdmap e204: 8 total, 8 up, 8 in 2024-02-05T07:54:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi131-82038-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi131-82038-33"}]': finished 2024-02-05T07:54:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:55 smithi131 ceph-mon[41654]: osdmap e205: 8 total, 8 up, 8 in 2024-02-05T07:54:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1718759709' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:55 smithi131 ceph-mon[29507]: osdmap e204: 8 total, 8 up, 8 in 2024-02-05T07:54:56.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi131-82038-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi131-82038-33"}]': finished 2024-02-05T07:54:56.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:55 smithi131 ceph-mon[29507]: osdmap e205: 8 total, 8 up, 8 in 2024-02-05T07:54:56.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1718759709' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:56.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:56.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:56.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:56 smithi157 ceph-mon[33345]: pgmap v268: 260 pgs: 260 active+clean; 583 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:54:57.268 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:56 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:57.268 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:57.268 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:57.268 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:56 smithi157 ceph-mon[33345]: osdmap e206: 8 total, 8 up, 8 in 2024-02-05T07:54:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:56 smithi131 ceph-mon[41654]: pgmap v268: 260 pgs: 260 active+clean; 583 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:54:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:56 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:56 smithi131 ceph-mon[41654]: osdmap e206: 8 total, 8 up, 8 in 2024-02-05T07:54:57.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:56 smithi131 ceph-mon[29507]: pgmap v268: 260 pgs: 260 active+clean; 583 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:54:57.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:56 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:54:57.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:56 smithi131 ceph-mon[29507]: osdmap e206: 8 total, 8 up, 8 in 2024-02-05T07:54:58.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:54:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:54:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:58 smithi157 ceph-mon[33345]: pgmap v271: 332 pgs: 72 unknown, 260 active+clean; 583 KiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:58 smithi157 ceph-mon[33345]: osdmap e207: 8 total, 8 up, 8 in 2024-02-05T07:54:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2530519839' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:58 smithi131 ceph-mon[41654]: pgmap v271: 332 pgs: 72 unknown, 260 active+clean; 583 KiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:58 smithi131 ceph-mon[41654]: osdmap e207: 8 total, 8 up, 8 in 2024-02-05T07:54:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2530519839' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:59.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:59.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:59.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:59.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:58 smithi131 ceph-mon[29507]: pgmap v271: 332 pgs: 72 unknown, 260 active+clean; 583 KiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:54:59.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:58 smithi131 ceph-mon[29507]: osdmap e207: 8 total, 8 up, 8 in 2024-02-05T07:54:59.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:59.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2530519839' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:59.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:59.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:54:59.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:59.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:54:59.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:59 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:59.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-82038-33"}]': finished 2024-02-05T07:54:59.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:54:59.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:54:59.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-23"}]: dispatch 2024-02-05T07:54:59.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:59 smithi157 ceph-mon[33345]: osdmap e208: 8 total, 8 up, 8 in 2024-02-05T07:54:59.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-23"}]: dispatch 2024-02-05T07:54:59.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:59.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:54:59.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:59 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:54:59.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:54:59 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-82038-33"}]': finished 2024-02-05T07:55:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-23"}]: dispatch 2024-02-05T07:55:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[41654]: osdmap e208: 8 total, 8 up, 8 in 2024-02-05T07:55:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-23"}]: dispatch 2024-02-05T07:55:00.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:55:00.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:55:00.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:55:00.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:00.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:00.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-82038-33"}]': finished 2024-02-05T07:55:00.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:00.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:00.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-23"}]: dispatch 2024-02-05T07:55:00.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[29507]: osdmap e208: 8 total, 8 up, 8 in 2024-02-05T07:55:00.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-23"}]: dispatch 2024-02-05T07:55:00.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1540323646' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:55:00.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-82038-33"}]: dispatch 2024-02-05T07:55:00.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:55:00.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:54:59 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:00.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:00 smithi157 ceph-mon[33345]: pgmap v274: 356 pgs: 32 creating+peering, 324 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-05T07:55:00.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-23"}]': finished 2024-02-05T07:55:00.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-82038-33"}]': finished 2024-02-05T07:55:00.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:00 smithi157 ceph-mon[33345]: osdmap e209: 8 total, 8 up, 8 in 2024-02-05T07:55:00.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-23", "mode": "writeback"}]: dispatch 2024-02-05T07:55:00.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1129531604' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:00.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-23", "mode": "writeback"}]: dispatch 2024-02-05T07:55:00.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-82038-34"}]: dispatch 2024-02-05T07:55:00.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-82038-34"}]: dispatch 2024-02-05T07:55:00.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi131-82038-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[41654]: pgmap v274: 356 pgs: 32 creating+peering, 324 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-05T07:55:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-23"}]': finished 2024-02-05T07:55:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-82038-33"}]': finished 2024-02-05T07:55:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[41654]: osdmap e209: 8 total, 8 up, 8 in 2024-02-05T07:55:01.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-23", "mode": "writeback"}]: dispatch 2024-02-05T07:55:01.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1129531604' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:01.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-23", "mode": "writeback"}]: dispatch 2024-02-05T07:55:01.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-82038-34"}]: dispatch 2024-02-05T07:55:01.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-82038-34"}]: dispatch 2024-02-05T07:55:01.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi131-82038-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:01.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[29507]: pgmap v274: 356 pgs: 32 creating+peering, 324 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-05T07:55:01.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-23"}]': finished 2024-02-05T07:55:01.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-82038-33"}]': finished 2024-02-05T07:55:01.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[29507]: osdmap e209: 8 total, 8 up, 8 in 2024-02-05T07:55:01.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-23", "mode": "writeback"}]: dispatch 2024-02-05T07:55:01.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1129531604' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:01.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-23", "mode": "writeback"}]: dispatch 2024-02-05T07:55:01.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-82038-34"}]: dispatch 2024-02-05T07:55:01.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-82038-34"}]: dispatch 2024-02-05T07:55:01.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi131-82038-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:01 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1129531604' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-23", "mode": "writeback"}]': finished 2024-02-05T07:55:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi131-82038-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:01 smithi157 ceph-mon[33345]: osdmap e210: 8 total, 8 up, 8 in 2024-02-05T07:55:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi131-82038-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi131-82038-34"}]: dispatch 2024-02-05T07:55:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:01 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1129531604' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-23", "mode": "writeback"}]': finished 2024-02-05T07:55:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi131-82038-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:01 smithi131 ceph-mon[41654]: osdmap e210: 8 total, 8 up, 8 in 2024-02-05T07:55:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi131-82038-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi131-82038-34"}]: dispatch 2024-02-05T07:55:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:01 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1129531604' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-23", "mode": "writeback"}]': finished 2024-02-05T07:55:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi131-82038-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:01 smithi131 ceph-mon[29507]: osdmap e210: 8 total, 8 up, 8 in 2024-02-05T07:55:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi131-82038-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi131-82038-34"}]: dispatch 2024-02-05T07:55:02.653 INFO:tasks.workunit.client.0.smithi131.stdout: 0x559251027870 2024-02-05T07:55:02.653 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510279a0 2024-02-05T07:55:02.653 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510280c0 2024-02-05T07:55:02.653 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251027c20 2024-02-05T07:55:02.653 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251027d30 2024-02-05T07:55:02.653 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251027e60 2024-02-05T07:55:02.653 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251027f70 2024-02-05T07:55:02.653 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251028690 2024-02-05T07:55:02.653 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510281f0 2024-02-05T07:55:02.653 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251028300 2024-02-05T07:55:02.654 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251028430 2024-02-05T07:55:02.654 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251028540 2024-02-05T07:55:02.654 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251028c60 2024-02-05T07:55:02.654 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510287c0 2024-02-05T07:55:02.654 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510288d0 2024-02-05T07:55:02.654 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251028a20 2024-02-05T07:55:02.654 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251028b30 2024-02-05T07:55:02.654 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251029210 2024-02-05T07:55:02.654 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251028d70 2024-02-05T07:55:02.654 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251028e80 2024-02-05T07:55:02.654 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251028fb0 2024-02-05T07:55:02.654 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510290c0 2024-02-05T07:55:02.654 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251029810 2024-02-05T07:55:02.654 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251029990 2024-02-05T07:55:02.654 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251029aa0 2024-02-05T07:55:02.654 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251029320 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251029450 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251029560 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251029690 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102a140 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251029ca0 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251029db0 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251029f00 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102a010 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102a6f0 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102a250 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102a380 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102a4b0 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102a5c0 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102ace0 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102a840 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102a970 2024-02-05T07:55:02.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102aa80 2024-02-05T07:55:02.656 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102abb0 2024-02-05T07:55:02.656 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102b290 2024-02-05T07:55:02.656 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102adf0 2024-02-05T07:55:02.656 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102af20 2024-02-05T07:55:02.656 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102b030 2024-02-05T07:55:02.656 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102b140 2024-02-05T07:55:02.656 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102b840 2024-02-05T07:55:02.656 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102b3f0 2024-02-05T07:55:02.656 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102b500 2024-02-05T07:55:02.656 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102b630 2024-02-05T07:55:02.656 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102b990 2024-02-05T07:55:02.656 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102be00 2024-02-05T07:55:02.656 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102bf10 2024-02-05T07:55:02.656 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102c4c0 2024-02-05T07:55:02.656 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102baa0 2024-02-05T07:55:02.657 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102bbd0 2024-02-05T07:55:02.657 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102bce0 2024-02-05T07:55:02.657 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102c5f0 2024-02-05T07:55:02.657 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102c020 2024-02-05T07:55:02.657 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102c170 2024-02-05T07:55:02.657 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102c280 2024-02-05T07:55:02.657 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102c390 2024-02-05T07:55:02.657 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102cbc0 2024-02-05T07:55:02.657 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102c720 2024-02-05T07:55:02.657 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102c830 2024-02-05T07:55:02.657 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102c940 2024-02-05T07:55:02.657 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102ca70 2024-02-05T07:55:02.657 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102d190 2024-02-05T07:55:02.657 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102ccf0 2024-02-05T07:55:02.657 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102ce70 2024-02-05T07:55:02.657 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102cff0 2024-02-05T07:55:02.658 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251024110 2024-02-05T07:55:02.658 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251024240 2024-02-05T07:55:02.658 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102db10 2024-02-05T07:55:02.658 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102d670 2024-02-05T07:55:02.658 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102d7c0 2024-02-05T07:55:02.658 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102d8d0 2024-02-05T07:55:02.658 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102da00 2024-02-05T07:55:02.658 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102e0c0 2024-02-05T07:55:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:02 smithi157 ceph-mon[33345]: pgmap v277: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-05T07:55:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:02 smithi157 ceph-mon[33345]: osdmap e211: 8 total, 8 up, 8 in 2024-02-05T07:55:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3233647367' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:03.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:03.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi131-82038-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi131-82038-34"}]': finished 2024-02-05T07:55:03.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:03.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:03.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:02 smithi157 ceph-mon[33345]: osdmap e212: 8 total, 8 up, 8 in 2024-02-05T07:55:03.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23"}]: dispatch 2024-02-05T07:55:03.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23"}]: dispatch 2024-02-05T07:55:03.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:55:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:55:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[29507]: pgmap v277: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-05T07:55:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[29507]: osdmap e211: 8 total, 8 up, 8 in 2024-02-05T07:55:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3233647367' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi131-82038-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi131-82038-34"}]': finished 2024-02-05T07:55:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[29507]: osdmap e212: 8 total, 8 up, 8 in 2024-02-05T07:55:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23"}]: dispatch 2024-02-05T07:55:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23"}]: dispatch 2024-02-05T07:55:03.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[41654]: pgmap v277: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-05T07:55:03.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[41654]: osdmap e211: 8 total, 8 up, 8 in 2024-02-05T07:55:03.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3233647367' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:03.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:03.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:03.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:03.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi131-82038-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi131-82038-34"}]': finished 2024-02-05T07:55:03.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:03.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:03.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[41654]: osdmap e212: 8 total, 8 up, 8 in 2024-02-05T07:55:03.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23"}]: dispatch 2024-02-05T07:55:03.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23"}]: dispatch 2024-02-05T07:55:04.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:03 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:04.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23"}]': finished 2024-02-05T07:55:04.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:03 smithi157 ceph-mon[33345]: osdmap e213: 8 total, 8 up, 8 in 2024-02-05T07:55:04.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1818800051' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:04.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:03 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23"}]': finished 2024-02-05T07:55:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:03 smithi131 ceph-mon[41654]: osdmap e213: 8 total, 8 up, 8 in 2024-02-05T07:55:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1818800051' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:04.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:03 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:04.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-23"}]': finished 2024-02-05T07:55:04.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:03 smithi131 ceph-mon[29507]: osdmap e213: 8 total, 8 up, 8 in 2024-02-05T07:55:04.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1818800051' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:04.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:04 smithi157 ceph-mon[33345]: pgmap v280: 428 pgs: 8 creating+peering, 420 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-05T07:55:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:04 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:04 smithi157 ceph-mon[33345]: osdmap e214: 8 total, 8 up, 8 in 2024-02-05T07:55:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-82038-34"}]: dispatch 2024-02-05T07:55:05.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:04 smithi131 ceph-mon[41654]: pgmap v280: 428 pgs: 8 creating+peering, 420 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-05T07:55:05.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:04 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:05.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:05.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:04 smithi131 ceph-mon[41654]: osdmap e214: 8 total, 8 up, 8 in 2024-02-05T07:55:05.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-82038-34"}]: dispatch 2024-02-05T07:55:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:04 smithi131 ceph-mon[29507]: pgmap v280: 428 pgs: 8 creating+peering, 420 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-05T07:55:05.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:04 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:05.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-82060-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:05.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:04 smithi131 ceph-mon[29507]: osdmap e214: 8 total, 8 up, 8 in 2024-02-05T07:55:05.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-82038-34"}]: dispatch 2024-02-05T07:55:06.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:06 smithi157 ceph-mon[33345]: pgmap v283: 420 pgs: 32 unknown, 388 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:55:06.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-82038-34"}]': finished 2024-02-05T07:55:06.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:06 smithi157 ceph-mon[33345]: osdmap e215: 8 total, 8 up, 8 in 2024-02-05T07:55:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-82038-34"}]: dispatch 2024-02-05T07:55:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-82038-34"}]': finished 2024-02-05T07:55:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:06 smithi157 ceph-mon[33345]: osdmap e216: 8 total, 8 up, 8 in 2024-02-05T07:55:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-82038-35"}]: dispatch 2024-02-05T07:55:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-82038-35"}]: dispatch 2024-02-05T07:55:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi131-82038-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[41654]: pgmap v283: 420 pgs: 32 unknown, 388 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:55:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-82038-34"}]': finished 2024-02-05T07:55:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[41654]: osdmap e215: 8 total, 8 up, 8 in 2024-02-05T07:55:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-82038-34"}]: dispatch 2024-02-05T07:55:07.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:07.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:07.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-82038-34"}]': finished 2024-02-05T07:55:07.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:07.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[41654]: osdmap e216: 8 total, 8 up, 8 in 2024-02-05T07:55:07.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-82038-35"}]: dispatch 2024-02-05T07:55:07.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-82038-35"}]: dispatch 2024-02-05T07:55:07.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi131-82038-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:07.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[29507]: pgmap v283: 420 pgs: 32 unknown, 388 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:55:07.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-82038-34"}]': finished 2024-02-05T07:55:07.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[29507]: osdmap e215: 8 total, 8 up, 8 in 2024-02-05T07:55:07.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-82038-34"}]: dispatch 2024-02-05T07:55:07.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:07.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:07.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3602110000' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-82038-34"}]': finished 2024-02-05T07:55:07.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:07.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[29507]: osdmap e216: 8 total, 8 up, 8 in 2024-02-05T07:55:07.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-82038-35"}]: dispatch 2024-02-05T07:55:07.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-82038-35"}]: dispatch 2024-02-05T07:55:07.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi131-82038-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:08.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:55:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:55:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:08 smithi157 ceph-mon[33345]: pgmap v286: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi131-82038-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:08 smithi157 ceph-mon[33345]: osdmap e217: 8 total, 8 up, 8 in 2024-02-05T07:55:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi131-82038-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi131-82038-35"}]: dispatch 2024-02-05T07:55:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:09.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:08 smithi131 ceph-mon[29507]: pgmap v286: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:09.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi131-82038-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:09.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:08 smithi131 ceph-mon[29507]: osdmap e217: 8 total, 8 up, 8 in 2024-02-05T07:55:09.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi131-82038-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi131-82038-35"}]: dispatch 2024-02-05T07:55:09.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:09.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:09.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:08 smithi131 ceph-mon[41654]: pgmap v286: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:09.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi131-82038-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:09.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:08 smithi131 ceph-mon[41654]: osdmap e217: 8 total, 8 up, 8 in 2024-02-05T07:55:09.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi131-82038-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi131-82038-35"}]: dispatch 2024-02-05T07:55:09.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:09.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-25"}]: dispatch 2024-02-05T07:55:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:09 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:09 smithi157 ceph-mon[33345]: osdmap e218: 8 total, 8 up, 8 in 2024-02-05T07:55:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-25"}]: dispatch 2024-02-05T07:55:10.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:09 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:55:10.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi131-82038-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi131-82038-35"}]': finished 2024-02-05T07:55:10.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-25"}]': finished 2024-02-05T07:55:10.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:09 smithi157 ceph-mon[33345]: osdmap e219: 8 total, 8 up, 8 in 2024-02-05T07:55:10.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-25", "mode": "writeback"}]: dispatch 2024-02-05T07:55:10.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-25", "mode": "writeback"}]: dispatch 2024-02-05T07:55:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-25"}]: dispatch 2024-02-05T07:55:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[29507]: osdmap e218: 8 total, 8 up, 8 in 2024-02-05T07:55:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-25"}]: dispatch 2024-02-05T07:55:10.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:55:10.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi131-82038-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi131-82038-35"}]': finished 2024-02-05T07:55:10.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-25"}]': finished 2024-02-05T07:55:10.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[29507]: osdmap e219: 8 total, 8 up, 8 in 2024-02-05T07:55:10.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-25", "mode": "writeback"}]: dispatch 2024-02-05T07:55:10.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-25", "mode": "writeback"}]: dispatch 2024-02-05T07:55:10.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:10.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-25"}]: dispatch 2024-02-05T07:55:10.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:10.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[41654]: osdmap e218: 8 total, 8 up, 8 in 2024-02-05T07:55:10.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-25"}]: dispatch 2024-02-05T07:55:10.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:55:10.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi131-82038-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi131-82038-35"}]': finished 2024-02-05T07:55:10.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-25"}]': finished 2024-02-05T07:55:10.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[41654]: osdmap e219: 8 total, 8 up, 8 in 2024-02-05T07:55:10.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-25", "mode": "writeback"}]: dispatch 2024-02-05T07:55:10.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-25", "mode": "writeback"}]: dispatch 2024-02-05T07:55:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:10 smithi157 ceph-mon[33345]: pgmap v289: 324 pgs: 324 active+clean; 584 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:55:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:10 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:55:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:10 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:55:11.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:10 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:11.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-25", "mode": "writeback"}]': finished 2024-02-05T07:55:11.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:10 smithi157 ceph-mon[33345]: osdmap e220: 8 total, 8 up, 8 in 2024-02-05T07:55:11.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3443643940' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi131-82060-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:11.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi131-82060-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:11.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:10 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:55:11.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:10 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:55:11.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:10 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:55:11.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:11.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[41654]: pgmap v289: 324 pgs: 324 active+clean; 584 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:55:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:55:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:55:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-25", "mode": "writeback"}]': finished 2024-02-05T07:55:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[41654]: osdmap e220: 8 total, 8 up, 8 in 2024-02-05T07:55:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3443643940' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi131-82060-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:11.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi131-82060-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:11.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:55:11.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:55:11.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:55:11.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:11.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[29507]: pgmap v289: 324 pgs: 324 active+clean; 584 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:55:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:55:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:55:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-25", "mode": "writeback"}]': finished 2024-02-05T07:55:11.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[29507]: osdmap e220: 8 total, 8 up, 8 in 2024-02-05T07:55:11.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3443643940' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi131-82060-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:11.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi131-82060-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:11.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:55:11.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:55:11.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:55:11.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:11.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:11 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi131-82060-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:12.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:12.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25"}]: dispatch 2024-02-05T07:55:12.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:11 smithi157 ceph-mon[33345]: osdmap e221: 8 total, 8 up, 8 in 2024-02-05T07:55:12.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25"}]: dispatch 2024-02-05T07:55:12.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-82038-35"}]: dispatch 2024-02-05T07:55:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:11 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi131-82060-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25"}]: dispatch 2024-02-05T07:55:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:11 smithi131 ceph-mon[41654]: osdmap e221: 8 total, 8 up, 8 in 2024-02-05T07:55:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25"}]: dispatch 2024-02-05T07:55:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-82038-35"}]: dispatch 2024-02-05T07:55:12.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:11 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:12.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi131-82060-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:12.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:12.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25"}]: dispatch 2024-02-05T07:55:12.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:11 smithi131 ceph-mon[29507]: osdmap e221: 8 total, 8 up, 8 in 2024-02-05T07:55:12.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25"}]: dispatch 2024-02-05T07:55:12.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-82038-35"}]: dispatch 2024-02-05T07:55:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:12 smithi157 ceph-mon[33345]: pgmap v292: 332 pgs: 40 unknown, 292 active+clean; 583 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:12 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25"}]': finished 2024-02-05T07:55:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-82038-35"}]': finished 2024-02-05T07:55:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:12 smithi157 ceph-mon[33345]: osdmap e222: 8 total, 8 up, 8 in 2024-02-05T07:55:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-82038-35"}]: dispatch 2024-02-05T07:55:13.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:55:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:55:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:55:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:12 smithi131 ceph-mon[41654]: pgmap v292: 332 pgs: 40 unknown, 292 active+clean; 583 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:12 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25"}]': finished 2024-02-05T07:55:13.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-82038-35"}]': finished 2024-02-05T07:55:13.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:12 smithi131 ceph-mon[41654]: osdmap e222: 8 total, 8 up, 8 in 2024-02-05T07:55:13.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-82038-35"}]: dispatch 2024-02-05T07:55:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:12 smithi131 ceph-mon[29507]: pgmap v292: 332 pgs: 40 unknown, 292 active+clean; 583 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:12 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-25"}]': finished 2024-02-05T07:55:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-82038-35"}]': finished 2024-02-05T07:55:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:12 smithi131 ceph-mon[29507]: osdmap e222: 8 total, 8 up, 8 in 2024-02-05T07:55:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-82038-35"}]: dispatch 2024-02-05T07:55:14.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:14 smithi157 ceph-mon[33345]: pgmap v295: 292 pgs: 292 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-05T07:55:14.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-82038-35"}]': finished 2024-02-05T07:55:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:14 smithi157 ceph-mon[33345]: osdmap e223: 8 total, 8 up, 8 in 2024-02-05T07:55:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2284069967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-82060-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-82060-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-82038-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-82038-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:14 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[41654]: pgmap v295: 292 pgs: 292 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-05T07:55:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-82038-35"}]': finished 2024-02-05T07:55:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[41654]: osdmap e223: 8 total, 8 up, 8 in 2024-02-05T07:55:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2284069967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-82060-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-82060-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-82038-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-82038-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:15.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[29507]: pgmap v295: 292 pgs: 292 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-05T07:55:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2352717858' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-82038-35"}]': finished 2024-02-05T07:55:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[29507]: osdmap e223: 8 total, 8 up, 8 in 2024-02-05T07:55:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2284069967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-82060-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-82060-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-82038-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-82038-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:15.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:14 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-82060-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-82038-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:15 smithi157 ceph-mon[33345]: osdmap e224: 8 total, 8 up, 8 in 2024-02-05T07:55:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-82038-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-82038-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:16.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:16.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-82060-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:16.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-82038-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:16.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:15 smithi131 ceph-mon[41654]: osdmap e224: 8 total, 8 up, 8 in 2024-02-05T07:55:16.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:16.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-82038-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:16.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:16.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-82038-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:16.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-82060-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-82038-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:15 smithi131 ceph-mon[29507]: osdmap e224: 8 total, 8 up, 8 in 2024-02-05T07:55:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-82038-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-82038-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:17.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[41654]: pgmap v298: 324 pgs: 64 unknown, 260 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:17.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[41654]: osdmap e225: 8 total, 8 up, 8 in 2024-02-05T07:55:17.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:17.171 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:17.171 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:17.171 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-82038-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-82038-36"}]': finished 2024-02-05T07:55:17.171 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:17.171 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[41654]: osdmap e226: 8 total, 8 up, 8 in 2024-02-05T07:55:17.171 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.171 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.171 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:17.171 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:17.171 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[29507]: pgmap v298: 324 pgs: 64 unknown, 260 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:17.171 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[29507]: osdmap e225: 8 total, 8 up, 8 in 2024-02-05T07:55:17.171 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:17.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:17.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:17.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-82038-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-82038-36"}]': finished 2024-02-05T07:55:17.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:17.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[29507]: osdmap e226: 8 total, 8 up, 8 in 2024-02-05T07:55:17.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:17.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:16 smithi157 ceph-mon[33345]: pgmap v298: 324 pgs: 64 unknown, 260 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:16 smithi157 ceph-mon[33345]: osdmap e225: 8 total, 8 up, 8 in 2024-02-05T07:55:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:17.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:17.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:16 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:17.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-82038-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-82038-36"}]': finished 2024-02-05T07:55:17.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:17.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:16 smithi157 ceph-mon[33345]: osdmap e226: 8 total, 8 up, 8 in 2024-02-05T07:55:17.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:17.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:17.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:18.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:55:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:55:18.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:18 smithi157 ceph-mon[33345]: pgmap v301: 300 pgs: 40 unknown, 260 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:18.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:18.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-27"}]: dispatch 2024-02-05T07:55:18.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:18 smithi157 ceph-mon[33345]: osdmap e227: 8 total, 8 up, 8 in 2024-02-05T07:55:18.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-27"}]: dispatch 2024-02-05T07:55:19.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:18 smithi131 ceph-mon[41654]: pgmap v301: 300 pgs: 40 unknown, 260 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:19.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:19.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-27"}]: dispatch 2024-02-05T07:55:19.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:18 smithi131 ceph-mon[41654]: osdmap e227: 8 total, 8 up, 8 in 2024-02-05T07:55:19.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-27"}]: dispatch 2024-02-05T07:55:19.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:18 smithi131 ceph-mon[29507]: pgmap v301: 300 pgs: 40 unknown, 260 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:19.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:19.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-27"}]: dispatch 2024-02-05T07:55:19.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:18 smithi131 ceph-mon[29507]: osdmap e227: 8 total, 8 up, 8 in 2024-02-05T07:55:19.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-27"}]: dispatch 2024-02-05T07:55:19.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:19 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:19.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-46"}]': finished 2024-02-05T07:55:19.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-27"}]': finished 2024-02-05T07:55:19.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-27", "mode": "writeback"}]: dispatch 2024-02-05T07:55:19.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:19 smithi157 ceph-mon[33345]: osdmap e228: 8 total, 8 up, 8 in 2024-02-05T07:55:19.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:19.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-27", "mode": "writeback"}]: dispatch 2024-02-05T07:55:19.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:20.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:19 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:20.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-46"}]': finished 2024-02-05T07:55:20.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-27"}]': finished 2024-02-05T07:55:20.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-27", "mode": "writeback"}]: dispatch 2024-02-05T07:55:20.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:19 smithi131 ceph-mon[41654]: osdmap e228: 8 total, 8 up, 8 in 2024-02-05T07:55:20.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:20.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-27", "mode": "writeback"}]: dispatch 2024-02-05T07:55:20.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:20.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:19 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:20.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-46"}]': finished 2024-02-05T07:55:20.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-27"}]': finished 2024-02-05T07:55:20.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-27", "mode": "writeback"}]: dispatch 2024-02-05T07:55:20.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:19 smithi131 ceph-mon[29507]: osdmap e228: 8 total, 8 up, 8 in 2024-02-05T07:55:20.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:20.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-27", "mode": "writeback"}]: dispatch 2024-02-05T07:55:20.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:20.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:20 smithi157 ceph-mon[33345]: pgmap v304: 300 pgs: 8 unknown, 292 active+clean; 584 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:55:20.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:20 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:20.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:20 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-27", "mode": "writeback"}]': finished 2024-02-05T07:55:20.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:20 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-82038-36"}]': finished 2024-02-05T07:55:20.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:20 smithi157 ceph-mon[33345]: osdmap e229: 8 total, 8 up, 8 in 2024-02-05T07:55:20.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:20 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:20.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:20 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:21.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:20 smithi131 ceph-mon[41654]: pgmap v304: 300 pgs: 8 unknown, 292 active+clean; 584 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:55:21.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:20 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:21.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:20 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-27", "mode": "writeback"}]': finished 2024-02-05T07:55:21.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:20 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-82038-36"}]': finished 2024-02-05T07:55:21.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:20 smithi131 ceph-mon[41654]: osdmap e229: 8 total, 8 up, 8 in 2024-02-05T07:55:21.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:20 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:21.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:20 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:21.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:20 smithi131 ceph-mon[29507]: pgmap v304: 300 pgs: 8 unknown, 292 active+clean; 584 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:55:21.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:20 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:21.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:20 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-27", "mode": "writeback"}]': finished 2024-02-05T07:55:21.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:20 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-82038-36"}]': finished 2024-02-05T07:55:21.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:20 smithi131 ceph-mon[29507]: osdmap e229: 8 total, 8 up, 8 in 2024-02-05T07:55:21.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:20 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/896710768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:21.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:20 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-82038-36"}]: dispatch 2024-02-05T07:55:21.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-82038-36"}]': finished 2024-02-05T07:55:21.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:21 smithi157 ceph-mon[33345]: osdmap e230: 8 total, 8 up, 8 in 2024-02-05T07:55:21.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:21.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:21.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:21.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:21.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:21.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:21.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:21.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:21.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:21.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:22.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-82038-36"}]': finished 2024-02-05T07:55:22.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[41654]: osdmap e230: 8 total, 8 up, 8 in 2024-02-05T07:55:22.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:22.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:22.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:22.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:22.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:22.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:22.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:22.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:22.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:22.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:22.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-82038-36"}]': finished 2024-02-05T07:55:22.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[29507]: osdmap e230: 8 total, 8 up, 8 in 2024-02-05T07:55:22.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:22.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:22.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:22.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:22.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:22.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:22.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:22.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:22.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:22.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:22 smithi157 ceph-mon[33345]: pgmap v307: 292 pgs: 292 active+clean; 584 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:55:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:22 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-46"}]': finished 2024-02-05T07:55:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:23.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:23.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27"}]: dispatch 2024-02-05T07:55:23.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:22 smithi157 ceph-mon[33345]: osdmap e231: 8 total, 8 up, 8 in 2024-02-05T07:55:23.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:23.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27"}]: dispatch 2024-02-05T07:55:23.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:23.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-82038-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:23.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-82038-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:55:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:55:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[41654]: pgmap v307: 292 pgs: 292 active+clean; 584 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:55:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-46"}]': finished 2024-02-05T07:55:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27"}]: dispatch 2024-02-05T07:55:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[41654]: osdmap e231: 8 total, 8 up, 8 in 2024-02-05T07:55:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27"}]: dispatch 2024-02-05T07:55:23.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:23.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-82038-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:23.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-82038-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[29507]: pgmap v307: 292 pgs: 292 active+clean; 584 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:55:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-46"}]': finished 2024-02-05T07:55:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27"}]: dispatch 2024-02-05T07:55:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[29507]: osdmap e231: 8 total, 8 up, 8 in 2024-02-05T07:55:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/443155540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27"}]: dispatch 2024-02-05T07:55:23.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-46"}]: dispatch 2024-02-05T07:55:23.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-82038-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:23.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-82038-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:24.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:23 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:24.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27"}]': finished 2024-02-05T07:55:24.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-46"}]': finished 2024-02-05T07:55:24.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:24.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:23 smithi157 ceph-mon[33345]: osdmap e232: 8 total, 8 up, 8 in 2024-02-05T07:55:24.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:24.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:24.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:24.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:24.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:24.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-82038-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-82038-37"}]': finished 2024-02-05T07:55:24.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:24.124 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:23 smithi157 ceph-mon[33345]: osdmap e233: 8 total, 8 up, 8 in 2024-02-05T07:55:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27"}]': finished 2024-02-05T07:55:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-46"}]': finished 2024-02-05T07:55:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[29507]: osdmap e232: 8 total, 8 up, 8 in 2024-02-05T07:55:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:24.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:24.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-82038-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-82038-37"}]': finished 2024-02-05T07:55:24.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:24.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[29507]: osdmap e233: 8 total, 8 up, 8 in 2024-02-05T07:55:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-27"}]': finished 2024-02-05T07:55:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-46"}]': finished 2024-02-05T07:55:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[41654]: osdmap e232: 8 total, 8 up, 8 in 2024-02-05T07:55:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:24.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:24.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-82038-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-82038-37"}]': finished 2024-02-05T07:55:24.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-82060-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:24.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:23 smithi131 ceph-mon[41654]: osdmap e233: 8 total, 8 up, 8 in 2024-02-05T07:55:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:24 smithi157 ceph-mon[33345]: pgmap v310: 292 pgs: 292 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T07:55:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:24 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:24 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:24 smithi157 ceph-mon[33345]: osdmap e234: 8 total, 8 up, 8 in 2024-02-05T07:55:25.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:25.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:24 smithi131 ceph-mon[29507]: pgmap v310: 292 pgs: 292 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T07:55:25.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:25.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:25.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:25.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:24 smithi131 ceph-mon[29507]: osdmap e234: 8 total, 8 up, 8 in 2024-02-05T07:55:25.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:25.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:24 smithi131 ceph-mon[41654]: pgmap v310: 292 pgs: 292 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T07:55:25.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:24 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:25.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:25.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:24 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:25.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:24 smithi131 ceph-mon[41654]: osdmap e234: 8 total, 8 up, 8 in 2024-02-05T07:55:25.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:26.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:26 smithi157 ceph-mon[33345]: pgmap v313: 300 pgs: 40 unknown, 260 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:26.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-47"}]': finished 2024-02-05T07:55:26.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:26 smithi157 ceph-mon[33345]: osdmap e235: 8 total, 8 up, 8 in 2024-02-05T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37"}]': finished 2024-02-05T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:26 smithi157 ceph-mon[33345]: osdmap e236: 8 total, 8 up, 8 in 2024-02-05T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:27.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[29507]: pgmap v313: 300 pgs: 40 unknown, 260 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:27.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-47"}]': finished 2024-02-05T07:55:27.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:27.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[29507]: osdmap e235: 8 total, 8 up, 8 in 2024-02-05T07:55:27.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:27.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:27.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37"}]': finished 2024-02-05T07:55:27.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[29507]: osdmap e236: 8 total, 8 up, 8 in 2024-02-05T07:55:27.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:27.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:27.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[41654]: pgmap v313: 300 pgs: 40 unknown, 260 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:27.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-82060-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-82060-47"}]': finished 2024-02-05T07:55:27.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:27.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[41654]: osdmap e235: 8 total, 8 up, 8 in 2024-02-05T07:55:27.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:27.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:27.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-82038-37"}]': finished 2024-02-05T07:55:27.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[41654]: osdmap e236: 8 total, 8 up, 8 in 2024-02-05T07:55:27.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2292413863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:27.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-82038-37"}]: dispatch 2024-02-05T07:55:28.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:55:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:55:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:28 smithi157 ceph-mon[33345]: pgmap v316: 300 pgs: 40 unknown, 260 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-82038-37"}]': finished 2024-02-05T07:55:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:28 smithi157 ceph-mon[33345]: osdmap e237: 8 total, 8 up, 8 in 2024-02-05T07:55:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-82038-38"}]: dispatch 2024-02-05T07:55:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-82038-38"}]: dispatch 2024-02-05T07:55:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi131-82038-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:28 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[41654]: pgmap v316: 300 pgs: 40 unknown, 260 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-82038-37"}]': finished 2024-02-05T07:55:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[41654]: osdmap e237: 8 total, 8 up, 8 in 2024-02-05T07:55:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:29.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:29.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-82038-38"}]: dispatch 2024-02-05T07:55:29.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-82038-38"}]: dispatch 2024-02-05T07:55:29.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi131-82038-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:29.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:29.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[29507]: pgmap v316: 300 pgs: 40 unknown, 260 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:29.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-82038-37"}]': finished 2024-02-05T07:55:29.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[29507]: osdmap e237: 8 total, 8 up, 8 in 2024-02-05T07:55:29.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:29.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:29.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:29.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:29.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-82038-38"}]: dispatch 2024-02-05T07:55:29.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-82038-38"}]: dispatch 2024-02-05T07:55:29.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi131-82038-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:29.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:28 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-47"}]': finished 2024-02-05T07:55:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi131-82038-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:29 smithi157 ceph-mon[33345]: osdmap e238: 8 total, 8 up, 8 in 2024-02-05T07:55:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-29"}]: dispatch 2024-02-05T07:55:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi131-82038-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi131-82038-38"}]: dispatch 2024-02-05T07:55:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-29"}]: dispatch 2024-02-05T07:55:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:29 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:29 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-29"}]': finished 2024-02-05T07:55:30.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-47"}]': finished 2024-02-05T07:55:30.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:30.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi131-82038-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:30.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[29507]: osdmap e238: 8 total, 8 up, 8 in 2024-02-05T07:55:30.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-29"}]: dispatch 2024-02-05T07:55:30.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:30.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi131-82038-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi131-82038-38"}]: dispatch 2024-02-05T07:55:30.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-29"}]: dispatch 2024-02-05T07:55:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-29"}]': finished 2024-02-05T07:55:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-82060-47"}]': finished 2024-02-05T07:55:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi131-82038-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[41654]: osdmap e238: 8 total, 8 up, 8 in 2024-02-05T07:55:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-29"}]: dispatch 2024-02-05T07:55:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2184564514' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi131-82038-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi131-82038-38"}]: dispatch 2024-02-05T07:55:30.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-29"}]: dispatch 2024-02-05T07:55:30.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-47"}]: dispatch 2024-02-05T07:55:30.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:30.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:30.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-29"}]': finished 2024-02-05T07:55:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: pgmap v319: 292 pgs: 292 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-02-05T07:55:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-47"}]': finished 2024-02-05T07:55:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-29", "mode": "writeback"}]: dispatch 2024-02-05T07:55:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: osdmap e239: 8 total, 8 up, 8 in 2024-02-05T07:55:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-29", "mode": "writeback"}]: dispatch 2024-02-05T07:55:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi131-82060-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi131-82060-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi131-82038-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi131-82038-38"}]': finished 2024-02-05T07:55:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-29", "mode": "writeback"}]': finished 2024-02-05T07:55:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi131-82060-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: osdmap e240: 8 total, 8 up, 8 in 2024-02-05T07:55:31.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi131-82060-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi131-82060-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: pgmap v319: 292 pgs: 292 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-02-05T07:55:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-47"}]': finished 2024-02-05T07:55:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-29", "mode": "writeback"}]: dispatch 2024-02-05T07:55:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: osdmap e239: 8 total, 8 up, 8 in 2024-02-05T07:55:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-29", "mode": "writeback"}]: dispatch 2024-02-05T07:55:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi131-82060-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi131-82060-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi131-82038-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi131-82038-38"}]': finished 2024-02-05T07:55:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-29", "mode": "writeback"}]': finished 2024-02-05T07:55:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi131-82060-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: osdmap e240: 8 total, 8 up, 8 in 2024-02-05T07:55:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi131-82060-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi131-82060-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: pgmap v319: 292 pgs: 292 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-02-05T07:55:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-82060-47"}]': finished 2024-02-05T07:55:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-29", "mode": "writeback"}]: dispatch 2024-02-05T07:55:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: osdmap e239: 8 total, 8 up, 8 in 2024-02-05T07:55:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-29", "mode": "writeback"}]: dispatch 2024-02-05T07:55:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi131-82060-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:31.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi131-82060-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:31.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:31.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi131-82038-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi131-82038-38"}]': finished 2024-02-05T07:55:31.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-29", "mode": "writeback"}]': finished 2024-02-05T07:55:31.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi131-82060-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:31.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: osdmap e240: 8 total, 8 up, 8 in 2024-02-05T07:55:31.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi131-82060-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:31.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi131-82060-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:32.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:32 smithi131 ceph-mon[41654]: pgmap v322: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-02-05T07:55:32.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:32 smithi131 ceph-mon[41654]: osdmap e241: 8 total, 8 up, 8 in 2024-02-05T07:55:32.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:32 smithi131 ceph-mon[29507]: pgmap v322: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-02-05T07:55:32.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:32 smithi131 ceph-mon[29507]: osdmap e241: 8 total, 8 up, 8 in 2024-02-05T07:55:32.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:32 smithi157 ceph-mon[33345]: pgmap v322: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-02-05T07:55:32.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:32 smithi157 ceph-mon[33345]: osdmap e241: 8 total, 8 up, 8 in 2024-02-05T07:55:33.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:55:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:55:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:55:33.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi131-82060-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi131-82060-48"}]': finished 2024-02-05T07:55:33.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:33 smithi157 ceph-mon[33345]: osdmap e242: 8 total, 8 up, 8 in 2024-02-05T07:55:33.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:33 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-82038-38"}]: dispatch 2024-02-05T07:55:33.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:33 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:33.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:34.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi131-82060-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi131-82060-48"}]': finished 2024-02-05T07:55:34.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:33 smithi131 ceph-mon[41654]: osdmap e242: 8 total, 8 up, 8 in 2024-02-05T07:55:34.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:33 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-82038-38"}]: dispatch 2024-02-05T07:55:34.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:33 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:34.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:34.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi131-82060-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi131-82060-48"}]': finished 2024-02-05T07:55:34.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:33 smithi131 ceph-mon[29507]: osdmap e242: 8 total, 8 up, 8 in 2024-02-05T07:55:34.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:33 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-82038-38"}]: dispatch 2024-02-05T07:55:34.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:33 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:34.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:34 smithi157 ceph-mon[33345]: pgmap v325: 300 pgs: 8 creating+peering, 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-02-05T07:55:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-82038-38"}]': finished 2024-02-05T07:55:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29"}]: dispatch 2024-02-05T07:55:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:34 smithi157 ceph-mon[33345]: osdmap e243: 8 total, 8 up, 8 in 2024-02-05T07:55:34.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-82038-38"}]: dispatch 2024-02-05T07:55:34.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29"}]: dispatch 2024-02-05T07:55:35.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:34 smithi131 ceph-mon[41654]: pgmap v325: 300 pgs: 8 creating+peering, 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-02-05T07:55:35.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-82038-38"}]': finished 2024-02-05T07:55:35.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:35.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29"}]: dispatch 2024-02-05T07:55:35.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:34 smithi131 ceph-mon[41654]: osdmap e243: 8 total, 8 up, 8 in 2024-02-05T07:55:35.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-82038-38"}]: dispatch 2024-02-05T07:55:35.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29"}]: dispatch 2024-02-05T07:55:35.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:34 smithi131 ceph-mon[29507]: pgmap v325: 300 pgs: 8 creating+peering, 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-02-05T07:55:35.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-82038-38"}]': finished 2024-02-05T07:55:35.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:35.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29"}]: dispatch 2024-02-05T07:55:35.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:34 smithi131 ceph-mon[29507]: osdmap e243: 8 total, 8 up, 8 in 2024-02-05T07:55:35.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-82038-38"}]: dispatch 2024-02-05T07:55:35.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29"}]: dispatch 2024-02-05T07:55:35.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:35 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:35.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-82038-38"}]': finished 2024-02-05T07:55:35.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29"}]': finished 2024-02-05T07:55:35.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:35 smithi157 ceph-mon[33345]: osdmap e244: 8 total, 8 up, 8 in 2024-02-05T07:55:35.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:35.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:35.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:35.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:35.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:35.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:35.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-82038-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:35.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-82038-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-82038-38"}]': finished 2024-02-05T07:55:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29"}]': finished 2024-02-05T07:55:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[41654]: osdmap e244: 8 total, 8 up, 8 in 2024-02-05T07:55:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:36.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:36.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:36.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:36.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-82038-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:36.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-82038-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1367947262' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-82038-38"}]': finished 2024-02-05T07:55:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-29"}]': finished 2024-02-05T07:55:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[29507]: osdmap e244: 8 total, 8 up, 8 in 2024-02-05T07:55:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:36.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:36.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:36.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:36.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-82038-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:36.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-82038-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:36 smithi157 ceph-mon[33345]: pgmap v328: 292 pgs: 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-02-05T07:55:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:36 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-82060-48"}]': finished 2024-02-05T07:55:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-82038-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-82038-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:36 smithi157 ceph-mon[33345]: osdmap e245: 8 total, 8 up, 8 in 2024-02-05T07:55:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-82038-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-82060-48"}]': finished 2024-02-05T07:55:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:36 smithi157 ceph-mon[33345]: osdmap e246: 8 total, 8 up, 8 in 2024-02-05T07:55:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[41654]: pgmap v328: 292 pgs: 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-02-05T07:55:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-82060-48"}]': finished 2024-02-05T07:55:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-82038-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-82038-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[41654]: osdmap e245: 8 total, 8 up, 8 in 2024-02-05T07:55:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:37.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-82038-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:37.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-82060-48"}]': finished 2024-02-05T07:55:37.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[41654]: osdmap e246: 8 total, 8 up, 8 in 2024-02-05T07:55:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[29507]: pgmap v328: 292 pgs: 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-02-05T07:55:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-82060-48"}]': finished 2024-02-05T07:55:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-82038-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1738994856' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-82038-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[29507]: osdmap e245: 8 total, 8 up, 8 in 2024-02-05T07:55:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-82060-48"}]: dispatch 2024-02-05T07:55:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-82038-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:37.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-82060-48"}]': finished 2024-02-05T07:55:37.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:36 smithi131 ceph-mon[29507]: osdmap e246: 8 total, 8 up, 8 in 2024-02-05T07:55:38.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:38.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.225 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi131-82060-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:38.225 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:38.225 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi131-82060-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:38.226 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-82038-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-82038-39"}]': finished 2024-02-05T07:55:38.226 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:38.226 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi131-82060-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:38.227 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:37 smithi157 ceph-mon[33345]: osdmap e247: 8 total, 8 up, 8 in 2024-02-05T07:55:38.227 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi131-82060-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.228 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:38.228 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi131-82060-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.229 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:38.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.510 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.511 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:38.511 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.511 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.511 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi131-82060-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:38.511 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:38.511 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi131-82060-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:38.511 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-82038-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-82038-39"}]': finished 2024-02-05T07:55:38.511 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:38.511 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi131-82060-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:38.511 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[29507]: osdmap e247: 8 total, 8 up, 8 in 2024-02-05T07:55:38.511 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi131-82060-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.511 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:38.512 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi131-82060-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.512 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:38.512 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.512 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.512 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:38.512 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.512 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.512 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi131-82060-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:38.512 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:38.512 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi131-82060-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:38.513 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-82038-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-82038-39"}]': finished 2024-02-05T07:55:38.513 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:38.513 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi131-82060-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:38.513 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[41654]: osdmap e247: 8 total, 8 up, 8 in 2024-02-05T07:55:38.513 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi131-82060-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.513 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:38.513 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi131-82060-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:38.513 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:38.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:55:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:55:38.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:38 smithi157 ceph-mon[33345]: pgmap v331: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:38.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:38.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:38 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-31"}]: dispatch 2024-02-05T07:55:38.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:38 smithi157 ceph-mon[33345]: osdmap e248: 8 total, 8 up, 8 in 2024-02-05T07:55:38.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-31"}]: dispatch 2024-02-05T07:55:39.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:38 smithi131 ceph-mon[29507]: pgmap v331: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:39.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:39.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:38 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-31"}]: dispatch 2024-02-05T07:55:39.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:38 smithi131 ceph-mon[29507]: osdmap e248: 8 total, 8 up, 8 in 2024-02-05T07:55:39.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-31"}]: dispatch 2024-02-05T07:55:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:38 smithi131 ceph-mon[41654]: pgmap v331: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:39.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:39.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:38 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-31"}]: dispatch 2024-02-05T07:55:39.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:38 smithi131 ceph-mon[41654]: osdmap e248: 8 total, 8 up, 8 in 2024-02-05T07:55:39.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-31"}]: dispatch 2024-02-05T07:55:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:39 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi131-82060-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi131-82060-49"}]': finished 2024-02-05T07:55:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-31"}]': finished 2024-02-05T07:55:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:39 smithi157 ceph-mon[33345]: osdmap e249: 8 total, 8 up, 8 in 2024-02-05T07:55:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:39 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-31", "mode": "writeback"}]: dispatch 2024-02-05T07:55:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-31", "mode": "writeback"}]: dispatch 2024-02-05T07:55:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:39 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:39 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi131-82060-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi131-82060-49"}]': finished 2024-02-05T07:55:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-31"}]': finished 2024-02-05T07:55:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:39 smithi131 ceph-mon[41654]: osdmap e249: 8 total, 8 up, 8 in 2024-02-05T07:55:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:39 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-31", "mode": "writeback"}]: dispatch 2024-02-05T07:55:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-31", "mode": "writeback"}]: dispatch 2024-02-05T07:55:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:39 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:39 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi131-82060-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi131-82060-49"}]': finished 2024-02-05T07:55:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-31"}]': finished 2024-02-05T07:55:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:39 smithi131 ceph-mon[29507]: osdmap e249: 8 total, 8 up, 8 in 2024-02-05T07:55:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-31", "mode": "writeback"}]: dispatch 2024-02-05T07:55:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-31", "mode": "writeback"}]: dispatch 2024-02-05T07:55:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:40 smithi157 ceph-mon[33345]: pgmap v334: 300 pgs: 8 creating+peering, 292 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:40 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-31", "mode": "writeback"}]': finished 2024-02-05T07:55:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-82038-39"}]': finished 2024-02-05T07:55:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:40 smithi157 ceph-mon[33345]: osdmap e250: 8 total, 8 up, 8 in 2024-02-05T07:55:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:40 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:40 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[41654]: pgmap v334: 300 pgs: 8 creating+peering, 292 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-31", "mode": "writeback"}]': finished 2024-02-05T07:55:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-82038-39"}]': finished 2024-02-05T07:55:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[41654]: osdmap e250: 8 total, 8 up, 8 in 2024-02-05T07:55:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:41.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[29507]: pgmap v334: 300 pgs: 8 creating+peering, 292 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-31", "mode": "writeback"}]': finished 2024-02-05T07:55:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-82038-39"}]': finished 2024-02-05T07:55:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[29507]: osdmap e250: 8 total, 8 up, 8 in 2024-02-05T07:55:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/438765458' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-82038-39"}]: dispatch 2024-02-05T07:55:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:41 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-82038-39"}]': finished 2024-02-05T07:55:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:42.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31"}]: dispatch 2024-02-05T07:55:42.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:41 smithi157 ceph-mon[33345]: osdmap e251: 8 total, 8 up, 8 in 2024-02-05T07:55:42.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:42.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31"}]: dispatch 2024-02-05T07:55:42.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:42.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:42.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:42.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:42.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:42.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-82038-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:42.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-82038-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-82038-39"}]': finished 2024-02-05T07:55:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31"}]: dispatch 2024-02-05T07:55:42.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[29507]: osdmap e251: 8 total, 8 up, 8 in 2024-02-05T07:55:42.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:42.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31"}]: dispatch 2024-02-05T07:55:42.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:42.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:42.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:42.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:42.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:42.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-82038-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:42.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-82038-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:42.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:42.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-82038-39"}]': finished 2024-02-05T07:55:42.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:42.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31"}]: dispatch 2024-02-05T07:55:42.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[41654]: osdmap e251: 8 total, 8 up, 8 in 2024-02-05T07:55:42.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:42.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31"}]: dispatch 2024-02-05T07:55:42.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:42.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:42.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:42.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:42.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:42.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-82038-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:42.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-82038-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:42 smithi157 ceph-mon[33345]: pgmap v337: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:42 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31"}]': finished 2024-02-05T07:55:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-82060-49"}]': finished 2024-02-05T07:55:43.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-82038-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:43.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:42 smithi157 ceph-mon[33345]: osdmap e252: 8 total, 8 up, 8 in 2024-02-05T07:55:43.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:42 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:43.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:43.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:42 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-82038-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:43.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-82038-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:43.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:55:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:55:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[41654]: pgmap v337: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31"}]': finished 2024-02-05T07:55:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-82060-49"}]': finished 2024-02-05T07:55:43.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-82038-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:43.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[41654]: osdmap e252: 8 total, 8 up, 8 in 2024-02-05T07:55:43.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:43.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:43.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-82038-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:43.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-82038-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[29507]: pgmap v337: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-31"}]': finished 2024-02-05T07:55:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-82060-49"}]': finished 2024-02-05T07:55:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-82038-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[29507]: osdmap e252: 8 total, 8 up, 8 in 2024-02-05T07:55:43.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1829710246' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:43.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-82060-49"}]: dispatch 2024-02-05T07:55:43.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-82038-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:43.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-82038-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:44.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:44 smithi157 ceph-mon[33345]: pgmap v340: 292 pgs: 292 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:55:44.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-82060-49"}]': finished 2024-02-05T07:55:44.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:44.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:44 smithi157 ceph-mon[33345]: osdmap e253: 8 total, 8 up, 8 in 2024-02-05T07:55:44.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:44.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:44.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:44.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-82060-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:44.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-82060-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:44.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:44 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[41654]: pgmap v340: 292 pgs: 292 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:55:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-82060-49"}]': finished 2024-02-05T07:55:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[41654]: osdmap e253: 8 total, 8 up, 8 in 2024-02-05T07:55:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-82060-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:45.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-82060-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:45.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:45.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[29507]: pgmap v340: 292 pgs: 292 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:55:45.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-82060-49"}]': finished 2024-02-05T07:55:45.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:45.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[29507]: osdmap e253: 8 total, 8 up, 8 in 2024-02-05T07:55:45.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:45.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:45.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:45.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-82060-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:45.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-82060-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:45.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:44 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:45.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-82038-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-82038-40"}]': finished 2024-02-05T07:55:45.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-82060-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:45.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:45 smithi157 ceph-mon[33345]: osdmap e254: 8 total, 8 up, 8 in 2024-02-05T07:55:45.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-82060-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:45.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:45.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:45.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-82060-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:46.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-82038-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-82038-40"}]': finished 2024-02-05T07:55:46.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-82060-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:46.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:45 smithi131 ceph-mon[41654]: osdmap e254: 8 total, 8 up, 8 in 2024-02-05T07:55:46.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-82060-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:46.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:46.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:46.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-82060-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:46.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-82038-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-82038-40"}]': finished 2024-02-05T07:55:46.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-82060-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:46.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:45 smithi131 ceph-mon[29507]: osdmap e254: 8 total, 8 up, 8 in 2024-02-05T07:55:46.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-82060-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:46.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:46.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:46.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-82060-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:46.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:46 smithi157 ceph-mon[33345]: pgmap v343: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:55:46.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:46 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:46.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:46 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:46.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:46 smithi157 ceph-mon[33345]: osdmap e255: 8 total, 8 up, 8 in 2024-02-05T07:55:46.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:46 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:46.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:46 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:47.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:46 smithi131 ceph-mon[41654]: pgmap v343: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:55:47.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:46 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:47.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:46 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:47.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:46 smithi131 ceph-mon[41654]: osdmap e255: 8 total, 8 up, 8 in 2024-02-05T07:55:47.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:46 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:47.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:46 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:47.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:46 smithi131 ceph-mon[29507]: pgmap v343: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:55:47.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:46 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:47.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:46 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:47.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:46 smithi131 ceph-mon[29507]: osdmap e255: 8 total, 8 up, 8 in 2024-02-05T07:55:47.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:46 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:47.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:46 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:47.919 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-82060-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-82060-50"}]': finished 2024-02-05T07:55:47.919 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:47.919 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:47 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-33"}]: dispatch 2024-02-05T07:55:47.920 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:47 smithi131 ceph-mon[41654]: osdmap e256: 8 total, 8 up, 8 in 2024-02-05T07:55:47.920 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-33"}]: dispatch 2024-02-05T07:55:47.920 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:47 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:47.920 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:47.920 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-82060-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-82060-50"}]': finished 2024-02-05T07:55:47.920 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:47.920 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:47 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-33"}]: dispatch 2024-02-05T07:55:47.920 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:47 smithi131 ceph-mon[29507]: osdmap e256: 8 total, 8 up, 8 in 2024-02-05T07:55:47.920 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-33"}]: dispatch 2024-02-05T07:55:47.920 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:47 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:47.920 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:47.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-82060-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-82060-50"}]': finished 2024-02-05T07:55:47.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:47.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:47 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-33"}]: dispatch 2024-02-05T07:55:47.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:47 smithi157 ceph-mon[33345]: osdmap e256: 8 total, 8 up, 8 in 2024-02-05T07:55:47.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-33"}]: dispatch 2024-02-05T07:55:47.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:47 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:47.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:48.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:55:48 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:55:48.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:48 smithi157 ceph-mon[33345]: pgmap v346: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:48.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-33"}]': finished 2024-02-05T07:55:48.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-82038-40"}]': finished 2024-02-05T07:55:48.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-33", "mode": "writeback"}]: dispatch 2024-02-05T07:55:48.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:48 smithi157 ceph-mon[33345]: osdmap e257: 8 total, 8 up, 8 in 2024-02-05T07:55:48.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:48.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-33", "mode": "writeback"}]: dispatch 2024-02-05T07:55:48.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:49.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:48 smithi131 ceph-mon[41654]: pgmap v346: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:49.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-33"}]': finished 2024-02-05T07:55:49.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-82038-40"}]': finished 2024-02-05T07:55:49.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-33", "mode": "writeback"}]: dispatch 2024-02-05T07:55:49.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:48 smithi131 ceph-mon[41654]: osdmap e257: 8 total, 8 up, 8 in 2024-02-05T07:55:49.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:49.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-33", "mode": "writeback"}]: dispatch 2024-02-05T07:55:49.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:49.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:48 smithi131 ceph-mon[29507]: pgmap v346: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:49.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-33"}]': finished 2024-02-05T07:55:49.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-82038-40"}]': finished 2024-02-05T07:55:49.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-33", "mode": "writeback"}]: dispatch 2024-02-05T07:55:49.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:48 smithi131 ceph-mon[29507]: osdmap e257: 8 total, 8 up, 8 in 2024-02-05T07:55:49.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1584635862' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:49.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-33", "mode": "writeback"}]: dispatch 2024-02-05T07:55:49.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-82038-40"}]: dispatch 2024-02-05T07:55:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:49 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:49 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-33", "mode": "writeback"}]': finished 2024-02-05T07:55:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-82038-40"}]': finished 2024-02-05T07:55:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:49 smithi157 ceph-mon[33345]: osdmap e258: 8 total, 8 up, 8 in 2024-02-05T07:55:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-82038-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-82038-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-33", "mode": "writeback"}]': finished 2024-02-05T07:55:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-82038-40"}]': finished 2024-02-05T07:55:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[41654]: osdmap e258: 8 total, 8 up, 8 in 2024-02-05T07:55:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:50.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:50.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:50.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:50.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:50.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-82038-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:50.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-82038-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:50.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:50.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:50.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:50.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:50.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-33", "mode": "writeback"}]': finished 2024-02-05T07:55:50.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-82038-40"}]': finished 2024-02-05T07:55:50.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[29507]: osdmap e258: 8 total, 8 up, 8 in 2024-02-05T07:55:50.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:50.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:50.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:50.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:50.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:50.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:50.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-82038-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:50.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-82038-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:50.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:50.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:51.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:51 smithi157 ceph-mon[33345]: pgmap v349: 292 pgs: 292 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:51.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-82060-50"}]': finished 2024-02-05T07:55:51.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-82038-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:51.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:51.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:51 smithi157 ceph-mon[33345]: osdmap e259: 8 total, 8 up, 8 in 2024-02-05T07:55:51.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33"}]: dispatch 2024-02-05T07:55:51.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:51.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33"}]: dispatch 2024-02-05T07:55:51.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:51.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-82038-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:51.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-82038-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:51.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:51 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:51.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33"}]': finished 2024-02-05T07:55:51.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[41654]: pgmap v349: 292 pgs: 292 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:51.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-82060-50"}]': finished 2024-02-05T07:55:51.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-82038-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:51.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:51.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[41654]: osdmap e259: 8 total, 8 up, 8 in 2024-02-05T07:55:51.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33"}]: dispatch 2024-02-05T07:55:51.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:51.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33"}]: dispatch 2024-02-05T07:55:51.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:51.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-82038-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:51.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-82038-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:51.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:51.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33"}]': finished 2024-02-05T07:55:51.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[29507]: pgmap v349: 292 pgs: 292 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:51.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-82060-50"}]': finished 2024-02-05T07:55:51.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-82038-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:51.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:55:51.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[29507]: osdmap e259: 8 total, 8 up, 8 in 2024-02-05T07:55:51.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33"}]: dispatch 2024-02-05T07:55:51.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2997763128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:51.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33"}]: dispatch 2024-02-05T07:55:51.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-82060-50"}]: dispatch 2024-02-05T07:55:51.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-82038-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:51.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-82038-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:51.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:55:51.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-33"}]': finished 2024-02-05T07:55:52.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-82060-50"}]': finished 2024-02-05T07:55:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:52 smithi157 ceph-mon[33345]: osdmap e260: 8 total, 8 up, 8 in 2024-02-05T07:55:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:52 smithi157 ceph-mon[33345]: pgmap v352: 292 pgs: 292 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:52 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-82038-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-82038-41"}]': finished 2024-02-05T07:55:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:52 smithi157 ceph-mon[33345]: osdmap e261: 8 total, 8 up, 8 in 2024-02-05T07:55:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2408701626' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-82060-50"}]': finished 2024-02-05T07:55:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-mon[41654]: osdmap e260: 8 total, 8 up, 8 in 2024-02-05T07:55:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-mon[41654]: pgmap v352: 292 pgs: 292 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-82038-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-82038-41"}]': finished 2024-02-05T07:55:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-mon[41654]: osdmap e261: 8 total, 8 up, 8 in 2024-02-05T07:55:52.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2408701626' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:52.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-82060-50"}]': finished 2024-02-05T07:55:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-mon[29507]: osdmap e260: 8 total, 8 up, 8 in 2024-02-05T07:55:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-mon[29507]: pgmap v352: 292 pgs: 292 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-82038-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-82038-41"}]': finished 2024-02-05T07:55:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-mon[29507]: osdmap e261: 8 total, 8 up, 8 in 2024-02-05T07:55:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2408701626' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:52.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:53.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:55:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:55:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:55:53.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:53.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:53 smithi157 ceph-mon[33345]: osdmap e262: 8 total, 8 up, 8 in 2024-02-05T07:55:53.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:53.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:53 smithi131 ceph-mon[41654]: osdmap e262: 8 total, 8 up, 8 in 2024-02-05T07:55:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:54.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-82060-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:54.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:53 smithi131 ceph-mon[29507]: osdmap e262: 8 total, 8 up, 8 in 2024-02-05T07:55:54.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:54.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:55:54.872 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:54 smithi157 ceph-mon[33345]: pgmap v355: 332 pgs: 32 creating+peering, 300 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:55:54.872 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:54.872 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:54 smithi157 ceph-mon[33345]: osdmap e263: 8 total, 8 up, 8 in 2024-02-05T07:55:54.872 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:54.872 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:54.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:54.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:54.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-82060-52"}]: dispatch 2024-02-05T07:55:54.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-82060-52"}]: dispatch 2024-02-05T07:55:54.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi131-82060-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:55.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[41654]: pgmap v355: 332 pgs: 32 creating+peering, 300 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:55:55.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:55.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[41654]: osdmap e263: 8 total, 8 up, 8 in 2024-02-05T07:55:55.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:55.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:55.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:55.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:55.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-82060-52"}]: dispatch 2024-02-05T07:55:55.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-82060-52"}]: dispatch 2024-02-05T07:55:55.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi131-82060-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:55.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[29507]: pgmap v355: 332 pgs: 32 creating+peering, 300 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:55:55.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:55:55.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[29507]: osdmap e263: 8 total, 8 up, 8 in 2024-02-05T07:55:55.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:55.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:55:55.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:55.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:55.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-82060-52"}]: dispatch 2024-02-05T07:55:55.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-82060-52"}]: dispatch 2024-02-05T07:55:55.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi131-82060-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-82038-41"}]': finished 2024-02-05T07:55:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi131-82060-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:55:56.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:55 smithi157 ceph-mon[33345]: osdmap e264: 8 total, 8 up, 8 in 2024-02-05T07:55:56.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:56.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi131-82060-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi131-82060-52"}]: dispatch 2024-02-05T07:55:56.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:55:56.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:56.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]': finished 2024-02-05T07:55:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-82038-41"}]': finished 2024-02-05T07:55:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi131-82060-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:55:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[41654]: osdmap e264: 8 total, 8 up, 8 in 2024-02-05T07:55:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi131-82060-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi131-82060-52"}]: dispatch 2024-02-05T07:55:56.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:55:56.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:56.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]': finished 2024-02-05T07:55:56.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:55:56.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-82038-41"}]': finished 2024-02-05T07:55:56.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi131-82060-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:56.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:55:56.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[29507]: osdmap e264: 8 total, 8 up, 8 in 2024-02-05T07:55:56.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4026609440' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:56.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi131-82060-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi131-82060-52"}]: dispatch 2024-02-05T07:55:56.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:55:56.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-82038-41"}]: dispatch 2024-02-05T07:55:56.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]': finished 2024-02-05T07:55:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: pgmap v358: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:55:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-82038-41"}]': finished 2024-02-05T07:55:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-35", "mode": "writeback"}]: dispatch 2024-02-05T07:55:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: osdmap e265: 8 total, 8 up, 8 in 2024-02-05T07:55:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-35", "mode": "writeback"}]: dispatch 2024-02-05T07:55:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-82038-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-82038-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi131-82060-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi131-82060-52"}]': finished 2024-02-05T07:55:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-35", "mode": "writeback"}]': finished 2024-02-05T07:55:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-82038-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:57.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: osdmap e266: 8 total, 8 up, 8 in 2024-02-05T07:55:57.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-82038-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-82038-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: pgmap v358: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:55:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-82038-41"}]': finished 2024-02-05T07:55:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-35", "mode": "writeback"}]: dispatch 2024-02-05T07:55:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: osdmap e265: 8 total, 8 up, 8 in 2024-02-05T07:55:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-35", "mode": "writeback"}]: dispatch 2024-02-05T07:55:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-82038-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-82038-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi131-82060-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi131-82060-52"}]': finished 2024-02-05T07:55:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-35", "mode": "writeback"}]': finished 2024-02-05T07:55:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-82038-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: osdmap e266: 8 total, 8 up, 8 in 2024-02-05T07:55:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-82038-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-82038-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: pgmap v358: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:55:57.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-82038-41"}]': finished 2024-02-05T07:55:57.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-35", "mode": "writeback"}]: dispatch 2024-02-05T07:55:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: osdmap e265: 8 total, 8 up, 8 in 2024-02-05T07:55:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-35", "mode": "writeback"}]: dispatch 2024-02-05T07:55:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-82038-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-82038-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:55:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:55:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi131-82060-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi131-82060-52"}]': finished 2024-02-05T07:55:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-35", "mode": "writeback"}]': finished 2024-02-05T07:55:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-82038-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:55:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: osdmap e266: 8 total, 8 up, 8 in 2024-02-05T07:55:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-82038-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:57.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-82038-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:55:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:58 smithi157 ceph-mon[33345]: pgmap v361: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:58 smithi157 ceph-mon[33345]: osdmap e267: 8 total, 8 up, 8 in 2024-02-05T07:55:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:58 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:58.972 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:55:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:55:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:58 smithi131 ceph-mon[41654]: pgmap v361: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:58 smithi131 ceph-mon[41654]: osdmap e267: 8 total, 8 up, 8 in 2024-02-05T07:55:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:58 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:59.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:58 smithi131 ceph-mon[29507]: pgmap v361: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:55:59.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:58 smithi131 ceph-mon[29507]: osdmap e267: 8 total, 8 up, 8 in 2024-02-05T07:55:59.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:58 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:55:59.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:59 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:59.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-82038-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-82038-42"}]': finished 2024-02-05T07:55:59.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:59 smithi157 ceph-mon[33345]: osdmap e268: 8 total, 8 up, 8 in 2024-02-05T07:55:59.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-82060-52"}]: dispatch 2024-02-05T07:55:59.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:59.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:55:59.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:59 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:55:59.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:55:59 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:59 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-82038-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-82038-42"}]': finished 2024-02-05T07:56:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:59 smithi131 ceph-mon[41654]: osdmap e268: 8 total, 8 up, 8 in 2024-02-05T07:56:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-82060-52"}]: dispatch 2024-02-05T07:56:00.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:00.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:00.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:59 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:56:00.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:55:59 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:00.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:59 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:00.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-82038-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-82038-42"}]': finished 2024-02-05T07:56:00.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:59 smithi131 ceph-mon[29507]: osdmap e268: 8 total, 8 up, 8 in 2024-02-05T07:56:00.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-82060-52"}]: dispatch 2024-02-05T07:56:00.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:00.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:00.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:59 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:56:00.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:55:59 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:00.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:00 smithi157 ceph-mon[33345]: pgmap v364: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:56:00.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-82060-52"}]': finished 2024-02-05T07:56:00.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:56:00.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:00 smithi157 ceph-mon[33345]: osdmap e269: 8 total, 8 up, 8 in 2024-02-05T07:56:00.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-82060-52"}]: dispatch 2024-02-05T07:56:00.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:56:00.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:56:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:00 smithi131 ceph-mon[41654]: pgmap v364: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:56:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-82060-52"}]': finished 2024-02-05T07:56:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:56:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:00 smithi131 ceph-mon[41654]: osdmap e269: 8 total, 8 up, 8 in 2024-02-05T07:56:01.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-82060-52"}]: dispatch 2024-02-05T07:56:01.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:56:01.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:56:01.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:00 smithi131 ceph-mon[29507]: pgmap v364: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:56:01.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-82060-52"}]': finished 2024-02-05T07:56:01.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:56:01.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:00 smithi131 ceph-mon[29507]: osdmap e269: 8 total, 8 up, 8 in 2024-02-05T07:56:01.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-82060-52"}]: dispatch 2024-02-05T07:56:01.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:56:01.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:56:01.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-82060-52"}]': finished 2024-02-05T07:56:01.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]': finished 2024-02-05T07:56:01.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: osdmap e270: 8 total, 8 up, 8 in 2024-02-05T07:56:01.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:56:01.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:56:01.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:01.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:01.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:01.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:01.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:01.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:01.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-82038-42"}]': finished 2024-02-05T07:56:01.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:01.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: osdmap e271: 8 total, 8 up, 8 in 2024-02-05T07:56:01.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:56:01.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-82060-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:01.973 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:56:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-82060-52"}]': finished 2024-02-05T07:56:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]': finished 2024-02-05T07:56:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: osdmap e270: 8 total, 8 up, 8 in 2024-02-05T07:56:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:56:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:56:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:02.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:02.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:02.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:02.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-82038-42"}]': finished 2024-02-05T07:56:02.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:02.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: osdmap e271: 8 total, 8 up, 8 in 2024-02-05T07:56:02.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:56:02.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-82060-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:02.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:56:02.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2299266902' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-82060-52"}]': finished 2024-02-05T07:56:02.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-35"}]': finished 2024-02-05T07:56:02.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: osdmap e270: 8 total, 8 up, 8 in 2024-02-05T07:56:02.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:56:02.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:56:02.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:02.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:02.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:02.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:02.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:02.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:02.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-82038-42"}]': finished 2024-02-05T07:56:02.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:02.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: osdmap e271: 8 total, 8 up, 8 in 2024-02-05T07:56:02.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3905695132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:56:02.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-82060-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:02.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-82038-42"}]: dispatch 2024-02-05T07:56:02.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: sta api_aio: [ OK ] LibRadosAioEC.SimpleWrite (6802 ms) 2024-02-05T07:56:02.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-02-05T07:56:02.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6951 ms) 2024-02-05T07:56:02.643 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-02-05T07:56:02.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7009 ms) 2024-02-05T07:56:02.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-02-05T07:56:02.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7006 ms) 2024-02-05T07:56:02.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-02-05T07:56:02.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7018 ms) 2024-02-05T07:56:02.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-02-05T07:56:02.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7026 ms) 2024-02-05T07:56:02.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-02-05T07:56:02.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7032 ms) 2024-02-05T07:56:02.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-02-05T07:56:02.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (6913 ms) 2024-02-05T07:56:02.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-02-05T07:56:02.644 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (7018 ms) 2024-02-05T07:56:02.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-02-05T07:56:02.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7025 ms) 2024-02-05T07:56:02.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-02-05T07:56:02.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (6963 ms) 2024-02-05T07:56:02.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-02-05T07:56:02.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (7028 ms) 2024-02-05T07:56:02.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-02-05T07:56:02.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6990 ms) 2024-02-05T07:56:02.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-02-05T07:56:02.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7028 ms) 2024-02-05T07:56:02.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-02-05T07:56:02.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7027 ms) 2024-02-05T07:56:02.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-02-05T07:56:02.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7026 ms) 2024-02-05T07:56:02.645 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (111863 ms total) 2024-02-05T07:56:02.646 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: 2024-02-05T07:56:02.646 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [----------] Global test environment tear-down 2024-02-05T07:56:02.646 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (203934 ms total) 2024-02-05T07:56:02.646 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ PASSED ] 42 tests. 2024-02-05T07:56:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:02 smithi157 ceph-mon[33345]: pgmap v367: 292 pgs: 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:56:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-82060-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-82038-42"}]': finished 2024-02-05T07:56:03.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:56:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:56:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:56:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:02 smithi131 ceph-mon[41654]: pgmap v367: 292 pgs: 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:56:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-82060-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-82038-42"}]': finished 2024-02-05T07:56:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:02 smithi131 ceph-mon[29507]: pgmap v367: 292 pgs: 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:56:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-82060-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-82038-42"}]': finished 2024-02-05T07:56:04.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:56:04.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:56:04.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:03 smithi157 ceph-mon[33345]: osdmap e272: 8 total, 8 up, 8 in 2024-02-05T07:56:04.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:56:04.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:03 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:56:04.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-82060-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-82060-53"}]': finished 2024-02-05T07:56:04.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35"}]': finished 2024-02-05T07:56:04.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:03 smithi157 ceph-mon[33345]: osdmap e273: 8 total, 8 up, 8 in 2024-02-05T07:56:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:56:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:56:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:03 smithi131 ceph-mon[41654]: osdmap e272: 8 total, 8 up, 8 in 2024-02-05T07:56:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:56:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:03 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:56:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-82060-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-82060-53"}]': finished 2024-02-05T07:56:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35"}]': finished 2024-02-05T07:56:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:03 smithi131 ceph-mon[41654]: osdmap e273: 8 total, 8 up, 8 in 2024-02-05T07:56:04.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:56:04.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:56:04.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:03 smithi131 ceph-mon[29507]: osdmap e272: 8 total, 8 up, 8 in 2024-02-05T07:56:04.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35"}]: dispatch 2024-02-05T07:56:04.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:03 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:56:04.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-82060-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-82060-53"}]': finished 2024-02-05T07:56:04.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-35"}]': finished 2024-02-05T07:56:04.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:03 smithi131 ceph-mon[29507]: osdmap e273: 8 total, 8 up, 8 in 2024-02-05T07:56:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:04 smithi157 ceph-mon[33345]: pgmap v370: 292 pgs: 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-05T07:56:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:04 smithi157 ceph-mon[33345]: osdmap e274: 8 total, 8 up, 8 in 2024-02-05T07:56:05.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:04 smithi131 ceph-mon[41654]: pgmap v370: 292 pgs: 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-05T07:56:05.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:04 smithi131 ceph-mon[41654]: osdmap e274: 8 total, 8 up, 8 in 2024-02-05T07:56:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:04 smithi131 ceph-mon[29507]: pgmap v370: 292 pgs: 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-05T07:56:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:04 smithi131 ceph-mon[29507]: osdmap e274: 8 total, 8 up, 8 in 2024-02-05T07:56:06.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:06 smithi157 ceph-mon[33345]: pgmap v373: 268 pgs: 8 unknown, 260 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:56:06.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:06 smithi157 ceph-mon[33345]: osdmap e275: 8 total, 8 up, 8 in 2024-02-05T07:56:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:06 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53"}]': finished 2024-02-05T07:56:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:06 smithi157 ceph-mon[33345]: osdmap e276: 8 total, 8 up, 8 in 2024-02-05T07:56:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[41654]: pgmap v373: 268 pgs: 8 unknown, 260 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:56:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[41654]: osdmap e275: 8 total, 8 up, 8 in 2024-02-05T07:56:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53"}]': finished 2024-02-05T07:56:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[41654]: osdmap e276: 8 total, 8 up, 8 in 2024-02-05T07:56:07.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:07.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:07.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:07.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:07.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[29507]: pgmap v373: 268 pgs: 8 unknown, 260 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:56:07.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[29507]: osdmap e275: 8 total, 8 up, 8 in 2024-02-05T07:56:07.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:07.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:07.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:07.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:07.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:07.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:07.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-82060-53"}]': finished 2024-02-05T07:56:07.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[29507]: osdmap e276: 8 total, 8 up, 8 in 2024-02-05T07:56:07.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3649162994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:07.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-82060-53"}]: dispatch 2024-02-05T07:56:07.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:07.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:08.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:56:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:56:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:08 smithi157 ceph-mon[33345]: pgmap v376: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-82060-53"}]': finished 2024-02-05T07:56:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:56:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:08 smithi157 ceph-mon[33345]: osdmap e277: 8 total, 8 up, 8 in 2024-02-05T07:56:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-37"}]: dispatch 2024-02-05T07:56:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-37"}]: dispatch 2024-02-05T07:56:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:08.973 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:08.973 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-82060-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:08.973 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-82060-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[29507]: pgmap v376: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-82060-53"}]': finished 2024-02-05T07:56:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:56:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[29507]: osdmap e277: 8 total, 8 up, 8 in 2024-02-05T07:56:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-37"}]: dispatch 2024-02-05T07:56:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-37"}]: dispatch 2024-02-05T07:56:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:09.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:09.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:09.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-82060-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:09.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-82060-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:09.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[41654]: pgmap v376: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:09.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-82060-53"}]': finished 2024-02-05T07:56:09.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:56:09.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[41654]: osdmap e277: 8 total, 8 up, 8 in 2024-02-05T07:56:09.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-37"}]: dispatch 2024-02-05T07:56:09.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-37"}]: dispatch 2024-02-05T07:56:09.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:09.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:09.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:09.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:09.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-82060-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:09.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-82060-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-37"}]': finished 2024-02-05T07:56:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-82060-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:09 smithi157 ceph-mon[33345]: osdmap e278: 8 total, 8 up, 8 in 2024-02-05T07:56:10.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-37", "mode": "writeback"}]: dispatch 2024-02-05T07:56:10.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-82060-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:10.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-37", "mode": "writeback"}]: dispatch 2024-02-05T07:56:10.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-82060-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:10.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:09 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:10.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:09 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:56:10.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-37", "mode": "writeback"}]': finished 2024-02-05T07:56:10.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-37"}]': finished 2024-02-05T07:56:10.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-82060-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:10.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[41654]: osdmap e278: 8 total, 8 up, 8 in 2024-02-05T07:56:10.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-37", "mode": "writeback"}]: dispatch 2024-02-05T07:56:10.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-82060-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:10.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-37", "mode": "writeback"}]: dispatch 2024-02-05T07:56:10.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-82060-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:10.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:10.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:56:10.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-37", "mode": "writeback"}]': finished 2024-02-05T07:56:10.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-37"}]': finished 2024-02-05T07:56:10.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-82060-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:10.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[29507]: osdmap e278: 8 total, 8 up, 8 in 2024-02-05T07:56:10.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-37", "mode": "writeback"}]: dispatch 2024-02-05T07:56:10.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-82060-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:10.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-37", "mode": "writeback"}]: dispatch 2024-02-05T07:56:10.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-82060-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:10.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:10.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:56:10.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-37", "mode": "writeback"}]': finished 2024-02-05T07:56:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:10 smithi157 ceph-mon[33345]: pgmap v379: 292 pgs: 292 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:10 smithi157 ceph-mon[33345]: osdmap e279: 8 total, 8 up, 8 in 2024-02-05T07:56:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-82060-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-82060-54"}]': finished 2024-02-05T07:56:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:56:11.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:10 smithi157 ceph-mon[33345]: osdmap e280: 8 total, 8 up, 8 in 2024-02-05T07:56:11.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37"}]: dispatch 2024-02-05T07:56:11.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37"}]: dispatch 2024-02-05T07:56:11.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:10 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:56:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[41654]: pgmap v379: 292 pgs: 292 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[41654]: osdmap e279: 8 total, 8 up, 8 in 2024-02-05T07:56:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-82060-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-82060-54"}]': finished 2024-02-05T07:56:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:56:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[41654]: osdmap e280: 8 total, 8 up, 8 in 2024-02-05T07:56:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37"}]: dispatch 2024-02-05T07:56:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37"}]: dispatch 2024-02-05T07:56:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:56:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[29507]: pgmap v379: 292 pgs: 292 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[29507]: osdmap e279: 8 total, 8 up, 8 in 2024-02-05T07:56:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-82060-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-82060-54"}]': finished 2024-02-05T07:56:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:56:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[29507]: osdmap e280: 8 total, 8 up, 8 in 2024-02-05T07:56:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37"}]: dispatch 2024-02-05T07:56:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37"}]: dispatch 2024-02-05T07:56:11.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:10 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:56:12.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:12 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:56:12.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:12 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:56:12.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:12 smithi157 ceph-mon[33345]: pgmap v382: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:12.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:12 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:12.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:12 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:56:12.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37"}]': finished 2024-02-05T07:56:12.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:12 smithi157 ceph-mon[33345]: osdmap e281: 8 total, 8 up, 8 in 2024-02-05T07:56:12.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:12 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:56:12.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:12 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:56:12.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:12 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:56:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:56:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:56:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[41654]: pgmap v382: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:56:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37"}]': finished 2024-02-05T07:56:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[41654]: osdmap e281: 8 total, 8 up, 8 in 2024-02-05T07:56:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:56:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:56:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:56:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:56:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:56:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[29507]: pgmap v382: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:56:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-37"}]': finished 2024-02-05T07:56:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[29507]: osdmap e281: 8 total, 8 up, 8 in 2024-02-05T07:56:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:56:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:56:12.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:56:13.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:56:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:56:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:56:13.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:13 smithi157 ceph-mon[33345]: osdmap e282: 8 total, 8 up, 8 in 2024-02-05T07:56:13.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:13 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:13.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:13 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:13 smithi131 ceph-mon[41654]: osdmap e282: 8 total, 8 up, 8 in 2024-02-05T07:56:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:13 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:13 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:14.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:13 smithi131 ceph-mon[29507]: osdmap e282: 8 total, 8 up, 8 in 2024-02-05T07:56:14.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:13 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:14.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:13 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:14.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:14 smithi157 ceph-mon[33345]: pgmap v385: 260 pgs: 260 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:56:14.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-82060-54"}]': finished 2024-02-05T07:56:14.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:14 smithi157 ceph-mon[33345]: osdmap e283: 8 total, 8 up, 8 in 2024-02-05T07:56:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:14 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:56:14.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:14 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[41654]: pgmap v385: 260 pgs: 260 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:56:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-82060-54"}]': finished 2024-02-05T07:56:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[41654]: osdmap e283: 8 total, 8 up, 8 in 2024-02-05T07:56:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:56:15.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:15.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[29507]: pgmap v385: 260 pgs: 260 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:56:15.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-82060-54"}]': finished 2024-02-05T07:56:15.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[29507]: osdmap e283: 8 total, 8 up, 8 in 2024-02-05T07:56:15.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/559731322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:15.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:15.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-82060-54"}]: dispatch 2024-02-05T07:56:15.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:15.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:56:15.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:14 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:15.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-82060-54"}]': finished 2024-02-05T07:56:15.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:15.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:15 smithi157 ceph-mon[33345]: osdmap e284: 8 total, 8 up, 8 in 2024-02-05T07:56:15.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:15.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:15.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-82060-55"}]: dispatch 2024-02-05T07:56:15.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-82060-55"}]: dispatch 2024-02-05T07:56:15.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi131-82060-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:15.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:56:15.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi131-82060-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:15.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:15 smithi157 ceph-mon[33345]: osdmap e285: 8 total, 8 up, 8 in 2024-02-05T07:56:15.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi131-82060-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi131-82060-55"}]: dispatch 2024-02-05T07:56:15.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-39"}]: dispatch 2024-02-05T07:56:15.973 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-39"}]: dispatch 2024-02-05T07:56:16.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-82060-54"}]': finished 2024-02-05T07:56:16.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:16.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[29507]: osdmap e284: 8 total, 8 up, 8 in 2024-02-05T07:56:16.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:16.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:16.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-82060-55"}]: dispatch 2024-02-05T07:56:16.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-82060-55"}]: dispatch 2024-02-05T07:56:16.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi131-82060-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:16.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:56:16.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi131-82060-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:16.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[29507]: osdmap e285: 8 total, 8 up, 8 in 2024-02-05T07:56:16.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi131-82060-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi131-82060-55"}]: dispatch 2024-02-05T07:56:16.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-39"}]: dispatch 2024-02-05T07:56:16.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-39"}]: dispatch 2024-02-05T07:56:16.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-82060-54"}]': finished 2024-02-05T07:56:16.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:16.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[41654]: osdmap e284: 8 total, 8 up, 8 in 2024-02-05T07:56:16.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:16.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:16.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-82060-55"}]: dispatch 2024-02-05T07:56:16.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-82060-55"}]: dispatch 2024-02-05T07:56:16.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi131-82060-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:16.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:56:16.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi131-82060-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:16.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[41654]: osdmap e285: 8 total, 8 up, 8 in 2024-02-05T07:56:16.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi131-82060-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi131-82060-55"}]: dispatch 2024-02-05T07:56:16.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-39"}]: dispatch 2024-02-05T07:56:16.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-39"}]: dispatch 2024-02-05T07:56:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:16 smithi157 ceph-mon[33345]: pgmap v388: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:56:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-39"}]': finished 2024-02-05T07:56:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:16 smithi131 ceph-mon[41654]: pgmap v388: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:56:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-39"}]': finished 2024-02-05T07:56:17.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:16 smithi131 ceph-mon[29507]: pgmap v388: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:56:17.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-39"}]': finished 2024-02-05T07:56:18.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-39", "mode": "writeback"}]: dispatch 2024-02-05T07:56:18.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[29507]: osdmap e286: 8 total, 8 up, 8 in 2024-02-05T07:56:18.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-39", "mode": "writeback"}]: dispatch 2024-02-05T07:56:18.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:56:18.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi131-82060-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi131-82060-55"}]': finished 2024-02-05T07:56:18.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-39", "mode": "writeback"}]': finished 2024-02-05T07:56:18.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[29507]: osdmap e287: 8 total, 8 up, 8 in 2024-02-05T07:56:18.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:18.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:18.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-39", "mode": "writeback"}]: dispatch 2024-02-05T07:56:18.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[41654]: osdmap e286: 8 total, 8 up, 8 in 2024-02-05T07:56:18.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-39", "mode": "writeback"}]: dispatch 2024-02-05T07:56:18.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:56:18.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi131-82060-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi131-82060-55"}]': finished 2024-02-05T07:56:18.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-39", "mode": "writeback"}]': finished 2024-02-05T07:56:18.171 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[41654]: osdmap e287: 8 total, 8 up, 8 in 2024-02-05T07:56:18.171 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:18.171 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-39", "mode": "writeback"}]: dispatch 2024-02-05T07:56:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:17 smithi157 ceph-mon[33345]: osdmap e286: 8 total, 8 up, 8 in 2024-02-05T07:56:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-39", "mode": "writeback"}]: dispatch 2024-02-05T07:56:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:17 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:56:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi131-82060-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi131-82060-55"}]': finished 2024-02-05T07:56:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-39", "mode": "writeback"}]': finished 2024-02-05T07:56:18.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:17 smithi157 ceph-mon[33345]: osdmap e287: 8 total, 8 up, 8 in 2024-02-05T07:56:18.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:18.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:18.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:56:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:56:18.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:18 smithi157 ceph-mon[33345]: pgmap v391: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:18.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:56:18.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:18 smithi157 ceph-mon[33345]: osdmap e288: 8 total, 8 up, 8 in 2024-02-05T07:56:18.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39"}]: dispatch 2024-02-05T07:56:18.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39"}]: dispatch 2024-02-05T07:56:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:18 smithi131 ceph-mon[29507]: pgmap v391: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:56:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:18 smithi131 ceph-mon[29507]: osdmap e288: 8 total, 8 up, 8 in 2024-02-05T07:56:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39"}]: dispatch 2024-02-05T07:56:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39"}]: dispatch 2024-02-05T07:56:19.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:18 smithi131 ceph-mon[41654]: pgmap v391: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:19.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:56:19.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:18 smithi131 ceph-mon[41654]: osdmap e288: 8 total, 8 up, 8 in 2024-02-05T07:56:19.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39"}]: dispatch 2024-02-05T07:56:19.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39"}]: dispatch 2024-02-05T07:56:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:19 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:19 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:56:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39"}]': finished 2024-02-05T07:56:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:19 smithi157 ceph-mon[33345]: osdmap e289: 8 total, 8 up, 8 in 2024-02-05T07:56:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-82060-55"}]: dispatch 2024-02-05T07:56:20.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:19 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:20.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:19 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:56:20.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39"}]': finished 2024-02-05T07:56:20.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:19 smithi131 ceph-mon[29507]: osdmap e289: 8 total, 8 up, 8 in 2024-02-05T07:56:20.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-82060-55"}]: dispatch 2024-02-05T07:56:20.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:19 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:20.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:19 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:56:20.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-39"}]': finished 2024-02-05T07:56:20.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:19 smithi131 ceph-mon[41654]: osdmap e289: 8 total, 8 up, 8 in 2024-02-05T07:56:20.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-82060-55"}]: dispatch 2024-02-05T07:56:20.597 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-02-05T07:56:20.598 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-02-05T07:56:20.598 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-02-05T07:56:20.598 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: seed 82290 2024-02-05T07:56:20.598 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-02-05T07:56:20.598 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (1410 ms) 2024-02-05T07:56:20.598 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-02-05T07:56:20.598 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (14696 ms) 2024-02-05T07:56:20.598 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-02-05T07:56:20.598 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (44 ms) 2024-02-05T07:56:20.598 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (16150 ms total) 2024-02-05T07:56:20.598 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: 2024-02-05T07:56:20.598 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-02-05T07:56:20.598 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-02-05T07:56:20.599 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (9002 ms) 2024-02-05T07:56:20.599 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-02-05T07:56:20.599 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (8009 ms) 2024-02-05T07:56:20.599 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-02-05T07:56:20.599 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (10009 ms) 2024-02-05T07:56:20.599 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-02-05T07:56:20.599 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: my_snaps [3] 2024-02-05T07:56:20.599 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: my_snaps [4,3] 2024-02-05T07:56:20.599 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: my_snaps [5,4,3] 2024-02-05T07:56:20.599 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-02-05T07:56:20.599 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: promoting some heads 2024-02-05T07:56:20.599 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: promoting from clones for snap 6 2024-02-05T07:56:20.599 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: promoting from clones for snap 5 2024-02-05T07:56:20.599 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: promoting from clones for snap 4 2024-02-05T07:56:20.599 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: promoting from clones for snap 3 2024-02-05T07:56:20.599 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: waiting for scrubs... 2024-02-05T07:56:20.600 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: done waiting 2024-02-05T07:56:20.600 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46066 ms) 2024-02-05T07:56:20.600 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-02-05T07:56:20.600 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9960 ms) 2024-02-05T07:56:20.600 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-02-05T07:56:20.600 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (8001 ms) 2024-02-05T07:56:20.600 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-02-05T07:56:20.600 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8019 ms) 2024-02-05T07:56:20.600 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-02-05T07:56:20.600 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8268 ms) 2024-02-05T07:56:20.600 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-02-05T07:56:20.600 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (9796 ms) 2024-02-05T07:56:20.600 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-02-05T07:56:20.600 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8925 ms) 2024-02-05T07:56:20.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-02-05T07:56:20.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10039 ms) 2024-02-05T07:56:20.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-02-05T07:56:20.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (11997 ms) 2024-02-05T07:56:20.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-02-05T07:56:20.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (8213 ms) 2024-02-05T07:56:20.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-02-05T07:56:20.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (7796 ms) 2024-02-05T07:56:20.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-02-05T07:56:20.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13044 ms) 2024-02-05T07:56:20.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-02-05T07:56:20.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7929 ms) 2024-02-05T07:56:20.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-02-05T07:56:20.601 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8029 ms) 2024-02-05T07:56:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:20 smithi157 ceph-mon[33345]: pgmap v394: 300 pgs: 8 creating+peering, 292 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 99 op/s 2024-02-05T07:56:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:20 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:20 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-82060-55"}]': finished 2024-02-05T07:56:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:20 smithi157 ceph-mon[33345]: osdmap e290: 8 total, 8 up, 8 in 2024-02-05T07:56:21.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:20 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-82060-55"}]: dispatch 2024-02-05T07:56:21.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:20 smithi131 ceph-mon[29507]: pgmap v394: 300 pgs: 8 creating+peering, 292 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 99 op/s 2024-02-05T07:56:21.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:20 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:21.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:20 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-82060-55"}]': finished 2024-02-05T07:56:21.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:20 smithi131 ceph-mon[29507]: osdmap e290: 8 total, 8 up, 8 in 2024-02-05T07:56:21.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:20 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-82060-55"}]: dispatch 2024-02-05T07:56:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:20 smithi131 ceph-mon[41654]: pgmap v394: 300 pgs: 8 creating+peering, 292 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 99 op/s 2024-02-05T07:56:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:20 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:21.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:20 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-82060-55"}]': finished 2024-02-05T07:56:21.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:20 smithi131 ceph-mon[41654]: osdmap e290: 8 total, 8 up, 8 in 2024-02-05T07:56:21.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:20 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-82060-55"}]: dispatch 2024-02-05T07:56:22.944 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[41654]: pgmap v397: 260 pgs: 260 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-05T07:56:22.944 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-82060-55"}]': finished 2024-02-05T07:56:22.944 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[41654]: osdmap e291: 8 total, 8 up, 8 in 2024-02-05T07:56:22.944 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:22.944 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:22.944 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:22.944 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:22.944 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:22.944 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:22.944 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-82060-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:22.945 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-82060-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:22.945 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[29507]: pgmap v397: 260 pgs: 260 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-05T07:56:22.945 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-82060-55"}]': finished 2024-02-05T07:56:22.945 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[29507]: osdmap e291: 8 total, 8 up, 8 in 2024-02-05T07:56:22.945 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:22.945 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:22.945 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:22.945 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:22.945 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:22.945 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:22.945 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-82060-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:22.946 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-82060-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:22.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:22 smithi157 ceph-mon[33345]: pgmap v397: 260 pgs: 260 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-05T07:56:22.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4254158481' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-82060-55"}]': finished 2024-02-05T07:56:22.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:22 smithi157 ceph-mon[33345]: osdmap e291: 8 total, 8 up, 8 in 2024-02-05T07:56:22.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:22.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:22.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:22.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:22.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:22.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:22.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-82060-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:22.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-82060-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:23.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:56:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:56:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:56:24.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:24.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-82060-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:24.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:23 smithi157 ceph-mon[33345]: osdmap e292: 8 total, 8 up, 8 in 2024-02-05T07:56:24.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:24.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:24.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-82060-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:24.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-82060-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:24.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:56:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-82060-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:23 smithi131 ceph-mon[29507]: osdmap e292: 8 total, 8 up, 8 in 2024-02-05T07:56:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:24.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-82060-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:24.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-82060-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:24.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:56:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-82060-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:23 smithi131 ceph-mon[41654]: osdmap e292: 8 total, 8 up, 8 in 2024-02-05T07:56:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-82060-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-82060-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:24.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:56:24.666 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSerted 0x55925102dc20 2024-02-05T07:56:24.667 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102dd50 2024-02-05T07:56:24.667 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102de80 2024-02-05T07:56:24.667 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102dfb0 2024-02-05T07:56:24.667 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102e670 2024-02-05T07:56:24.667 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102e1d0 2024-02-05T07:56:24.667 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102d2c0 2024-02-05T07:56:24.667 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102ef80 2024-02-05T07:56:24.667 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102eb30 2024-02-05T07:56:24.668 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102d3f0 2024-02-05T07:56:24.668 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102d500 2024-02-05T07:56:24.668 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102ec40 2024-02-05T07:56:24.668 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102ed70 2024-02-05T07:56:24.668 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102f0d0 2024-02-05T07:56:24.668 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102f580 2024-02-05T07:56:24.668 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102fa30 2024-02-05T07:56:24.668 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102fb40 2024-02-05T07:56:24.668 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102f1e0 2024-02-05T07:56:24.668 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102f330 2024-02-05T07:56:24.668 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102f460 2024-02-05T07:56:24.669 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102e780 2024-02-05T07:56:24.669 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102e8b0 2024-02-05T07:56:24.669 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102ea00 2024-02-05T07:56:24.669 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251030110 2024-02-05T07:56:24.669 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102fc70 2024-02-05T07:56:24.669 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102fd80 2024-02-05T07:56:24.669 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102feb0 2024-02-05T07:56:24.669 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102ffe0 2024-02-05T07:56:24.669 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251030700 2024-02-05T07:56:24.669 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251030260 2024-02-05T07:56:24.669 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251030390 2024-02-05T07:56:24.669 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510304c0 2024-02-05T07:56:24.670 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510305f0 2024-02-05T07:56:24.670 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251030cd0 2024-02-05T07:56:24.670 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251030830 2024-02-05T07:56:24.670 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251030960 2024-02-05T07:56:24.670 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251030a70 2024-02-05T07:56:24.670 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251030ba0 2024-02-05T07:56:24.670 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510312a0 2024-02-05T07:56:24.670 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251030e00 2024-02-05T07:56:24.670 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251030f30 2024-02-05T07:56:24.670 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251031040 2024-02-05T07:56:24.670 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251031150 2024-02-05T07:56:24.670 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251031870 2024-02-05T07:56:24.670 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510313d0 2024-02-05T07:56:24.670 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510314e0 2024-02-05T07:56:24.670 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251031610 2024-02-05T07:56:24.671 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251031740 2024-02-05T07:56:24.671 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251031e40 2024-02-05T07:56:24.671 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510319a0 2024-02-05T07:56:24.671 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251031af0 2024-02-05T07:56:24.671 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251031c00 2024-02-05T07:56:24.671 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251031f70 2024-02-05T07:56:24.671 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251032400 2024-02-05T07:56:24.671 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251032530 2024-02-05T07:56:24.671 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251032080 2024-02-05T07:56:24.671 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510321d0 2024-02-05T07:56:24.671 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251032b20 2024-02-05T07:56:24.671 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251032680 2024-02-05T07:56:24.671 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510327b0 2024-02-05T07:56:24.671 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510328e0 2024-02-05T07:56:24.671 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251032a10 2024-02-05T07:56:24.672 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510330f0 2024-02-05T07:56:24.672 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251032c50 2024-02-05T07:56:24.672 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251032d80 2024-02-05T07:56:24.672 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251032e90 2024-02-05T07:56:24.672 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251032fe0 2024-02-05T07:56:24.672 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510336c0 2024-02-05T07:56:24.672 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251033220 2024-02-05T07:56:24.672 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251033370 2024-02-05T07:56:24.673 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251033480 2024-02-05T07:56:24.673 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510335b0 2024-02-05T07:56:24.673 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251033c90 2024-02-05T07:56:24.673 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510337f0 2024-02-05T07:56:24.673 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251033900 2024-02-05T07:56:24.673 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251033a50 2024-02-05T07:56:24.673 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251033de0 2024-02-05T07:56:24.673 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251034290 2024-02-05T07:56:24.674 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510343c0 2024-02-05T07:56:24.674 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251034990 2024-02-05T07:56:24.674 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251033f30 2024-02-05T07:56:24.674 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251034040 2024-02-05T07:56:25.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:24 smithi157 ceph-mon[33345]: pgmap v400: 292 pgs: 292 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:25.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:24 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:56:25.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:24 smithi157 ceph-mon[33345]: osdmap e293: 8 total, 8 up, 8 in 2024-02-05T07:56:25.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:56:25.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-82060-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-82060-56"}]': finished 2024-02-05T07:56:25.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:56:25.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:24 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:56:25.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:24 smithi157 ceph-mon[33345]: osdmap e294: 8 total, 8 up, 8 in 2024-02-05T07:56:25.124 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:56:25.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[41654]: pgmap v400: 292 pgs: 292 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:25.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:56:25.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[41654]: osdmap e293: 8 total, 8 up, 8 in 2024-02-05T07:56:25.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:56:25.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-82060-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-82060-56"}]': finished 2024-02-05T07:56:25.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:56:25.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:56:25.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[41654]: osdmap e294: 8 total, 8 up, 8 in 2024-02-05T07:56:25.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:56:25.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[29507]: pgmap v400: 292 pgs: 292 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:25.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:56:25.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[29507]: osdmap e293: 8 total, 8 up, 8 in 2024-02-05T07:56:25.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:56:25.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-82060-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-82060-56"}]': finished 2024-02-05T07:56:25.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:56:25.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:56:25.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[29507]: osdmap e294: 8 total, 8 up, 8 in 2024-02-05T07:56:25.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:56:26.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:26 smithi157 ceph-mon[33345]: pgmap v403: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:26.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:56:26.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:26 smithi157 ceph-mon[33345]: osdmap e295: 8 total, 8 up, 8 in 2024-02-05T07:56:26.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:56:26.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:56:26.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:26 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:27.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:26 smithi131 ceph-mon[41654]: pgmap v403: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:27.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:56:27.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:26 smithi131 ceph-mon[41654]: osdmap e295: 8 total, 8 up, 8 in 2024-02-05T07:56:27.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:56:27.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:56:27.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:26 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:27.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:26 smithi131 ceph-mon[29507]: pgmap v403: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:27.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:56:27.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:26 smithi131 ceph-mon[29507]: osdmap e295: 8 total, 8 up, 8 in 2024-02-05T07:56:27.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:56:27.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:56:27.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:26 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:27.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T07:56:27.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:27 smithi157 ceph-mon[33345]: osdmap e296: 8 total, 8 up, 8 in 2024-02-05T07:56:27.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:27 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:27.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:27.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-82060-56"}]': finished 2024-02-05T07:56:27.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:27 smithi157 ceph-mon[33345]: osdmap e297: 8 total, 8 up, 8 in 2024-02-05T07:56:28.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T07:56:28.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:27 smithi131 ceph-mon[41654]: osdmap e296: 8 total, 8 up, 8 in 2024-02-05T07:56:28.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:27 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:28.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:28.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-82060-56"}]': finished 2024-02-05T07:56:28.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:27 smithi131 ceph-mon[41654]: osdmap e297: 8 total, 8 up, 8 in 2024-02-05T07:56:28.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T07:56:28.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:27 smithi131 ceph-mon[29507]: osdmap e296: 8 total, 8 up, 8 in 2024-02-05T07:56:28.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:27 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:28.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:28.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-82060-56"}]': finished 2024-02-05T07:56:28.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:27 smithi131 ceph-mon[29507]: osdmap e297: 8 total, 8 up, 8 in 2024-02-05T07:56:28.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:56:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:56:28.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:28 smithi157 ceph-mon[33345]: pgmap v406: 292 pgs: 292 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41"}]: dispatch 2024-02-05T07:56:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41"}]: dispatch 2024-02-05T07:56:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-82060-56"}]': finished 2024-02-05T07:56:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41"}]': finished 2024-02-05T07:56:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:28 smithi157 ceph-mon[33345]: osdmap e298: 8 total, 8 up, 8 in 2024-02-05T07:56:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[41654]: pgmap v406: 292 pgs: 292 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41"}]: dispatch 2024-02-05T07:56:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41"}]: dispatch 2024-02-05T07:56:29.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-82060-56"}]': finished 2024-02-05T07:56:29.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41"}]': finished 2024-02-05T07:56:29.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[41654]: osdmap e298: 8 total, 8 up, 8 in 2024-02-05T07:56:29.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[29507]: pgmap v406: 292 pgs: 292 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:29.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3980634209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:29.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-82060-56"}]: dispatch 2024-02-05T07:56:29.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:29.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:29.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41"}]: dispatch 2024-02-05T07:56:29.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41"}]: dispatch 2024-02-05T07:56:29.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-82060-56"}]': finished 2024-02-05T07:56:29.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-41"}]': finished 2024-02-05T07:56:29.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:28 smithi131 ceph-mon[29507]: osdmap e298: 8 total, 8 up, 8 in 2024-02-05T07:56:29.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:29 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:29.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:29.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:29.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:29.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:29.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-82060-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:29.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-82060-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:29.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:29 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:30.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:29 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:30.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:30.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:30.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:30.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:30.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-82060-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:30.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-82060-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:30.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:29 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:30.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:29 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:30.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:30.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:30.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:30.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:30.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-82060-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:30.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-82060-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:30.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:29 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:30 smithi157 ceph-mon[33345]: pgmap v409: 292 pgs: 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-82060-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:30 smithi157 ceph-mon[33345]: osdmap e299: 8 total, 8 up, 8 in 2024-02-05T07:56:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-82060-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-82060-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:30 smithi131 ceph-mon[41654]: pgmap v409: 292 pgs: 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-82060-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:30 smithi131 ceph-mon[41654]: osdmap e299: 8 total, 8 up, 8 in 2024-02-05T07:56:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-82060-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-82060-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:30 smithi131 ceph-mon[29507]: pgmap v409: 292 pgs: 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-82060-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:31.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:30 smithi131 ceph-mon[29507]: osdmap e299: 8 total, 8 up, 8 in 2024-02-05T07:56:31.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-82060-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:31.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-82060-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:32.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:31 smithi157 ceph-mon[33345]: osdmap e300: 8 total, 8 up, 8 in 2024-02-05T07:56:32.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:31 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:32.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:31 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:32.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:31 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:32.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:31 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-82060-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-82060-57"}]': finished 2024-02-05T07:56:32.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:31 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:32.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:31 smithi157 ceph-mon[33345]: osdmap e301: 8 total, 8 up, 8 in 2024-02-05T07:56:32.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:31 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi131-82290-6","var": "pg_num","format": "json"}]: dispatch 2024-02-05T07:56:32.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:31 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:32.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:31 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:32.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[29507]: osdmap e300: 8 total, 8 up, 8 in 2024-02-05T07:56:32.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:32.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:32.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:32.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-82060-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-82060-57"}]': finished 2024-02-05T07:56:32.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:32.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[29507]: osdmap e301: 8 total, 8 up, 8 in 2024-02-05T07:56:32.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi131-82290-6","var": "pg_num","format": "json"}]: dispatch 2024-02-05T07:56:32.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:32.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:32.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[41654]: osdmap e300: 8 total, 8 up, 8 in 2024-02-05T07:56:32.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:32.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:32.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:32.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-82060-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-82060-57"}]': finished 2024-02-05T07:56:32.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:32.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[41654]: osdmap e301: 8 total, 8 up, 8 in 2024-02-05T07:56:32.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi131-82290-6","var": "pg_num","format": "json"}]: dispatch 2024-02-05T07:56:32.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:32.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:31 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:32 smithi157 ceph-mon[33345]: pgmap v412: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:56:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:32 smithi157 ceph-mon[33345]: osdmap e302: 8 total, 8 up, 8 in 2024-02-05T07:56:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:32 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-05T07:56:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-05T07:56:33.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:56:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:56:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:56:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:32 smithi131 ceph-mon[41654]: pgmap v412: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:56:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:32 smithi131 ceph-mon[41654]: osdmap e302: 8 total, 8 up, 8 in 2024-02-05T07:56:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:32 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-05T07:56:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-05T07:56:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:32 smithi131 ceph-mon[29507]: pgmap v412: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:56:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:32 smithi131 ceph-mon[29507]: osdmap e302: 8 total, 8 up, 8 in 2024-02-05T07:56:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:32 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-05T07:56:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-05T07:56:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:34 smithi157 ceph-mon[33345]: pgmap v415: 300 pgs: 300 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:56:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_count","val": "8"}]': finished 2024-02-05T07:56:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:34 smithi157 ceph-mon[33345]: osdmap e303: 8 total, 8 up, 8 in 2024-02-05T07:56:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:56:34.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:56:34.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:34.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:35.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:34 smithi131 ceph-mon[29507]: pgmap v415: 300 pgs: 300 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:56:35.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_count","val": "8"}]': finished 2024-02-05T07:56:35.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:34 smithi131 ceph-mon[29507]: osdmap e303: 8 total, 8 up, 8 in 2024-02-05T07:56:35.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:56:35.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:56:35.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:35.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:35.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:34 smithi131 ceph-mon[41654]: pgmap v415: 300 pgs: 300 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:56:35.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_count","val": "8"}]': finished 2024-02-05T07:56:35.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:34 smithi131 ceph-mon[41654]: osdmap e303: 8 total, 8 up, 8 in 2024-02-05T07:56:35.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:56:35.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:56:35.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:35.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:35.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc:tRead 2024-02-05T07:56:35.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-05T07:56:35.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: ok, hit_set contains 274:602f83fe:::foo:head 2024-02-05T07:56:35.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9033 ms) 2024-02-05T07:56:35.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-02-05T07:56:35.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg_num = 32 2024-02-05T07:56:35.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 0 ls 1707119796,0 2024-02-05T07:56:35.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 1 ls 1707119796,0 2024-02-05T07:56:35.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 2 ls 1707119796,0 2024-02-05T07:56:35.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 3 ls 1707119796,0 2024-02-05T07:56:35.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 4 ls 1707119796,0 2024-02-05T07:56:35.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 5 ls 1707119796,0 2024-02-05T07:56:35.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 6 ls 1707119796,0 2024-02-05T07:56:35.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 7 ls 1707119796,0 2024-02-05T07:56:35.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 8 ls 1707119796,0 2024-02-05T07:56:35.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 9 ls 1707119796,0 2024-02-05T07:56:35.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 10 ls 1707119796,0 2024-02-05T07:56:35.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 11 ls 1707119796,0 2024-02-05T07:56:35.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 12 ls 1707119796,0 2024-02-05T07:56:35.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 13 ls 1707119796,0 2024-02-05T07:56:35.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 14 ls 1707119796,0 2024-02-05T07:56:35.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 15 ls 1707119796,0 2024-02-05T07:56:35.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 16 ls 1707119796,0 2024-02-05T07:56:35.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 17 ls 1707119796,0 2024-02-05T07:56:35.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 18 ls 1707119796,0 2024-02-05T07:56:35.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 19 ls 1707119796,0 2024-02-05T07:56:35.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 20 ls 1707119796,0 2024-02-05T07:56:35.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 21 ls 1707119796,0 2024-02-05T07:56:35.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 22 ls 1707119796,0 2024-02-05T07:56:35.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 23 ls 1707119796,0 2024-02-05T07:56:35.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 24 ls 1707119796,0 2024-02-05T07:56:35.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 25 ls 1707119796,0 2024-02-05T07:56:35.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 26 ls 1707119796,0 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 27 ls 1707119796,0 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 28 ls 1707119796,0 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 29 ls 1707119796,0 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 30 ls 1707119796,0 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 31 ls 1707119796,0 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg_num = 32 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:6cac518f:::0:head 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:02547ec2:::1:head 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:f905c69b:::2:head 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:cfc208b3:::3:head 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:d83876eb:::4:head 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:b29083e3:::5:head 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:c4fdafeb:::6:head 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:5c6b0b28:::7:head 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:bd63b0f1:::8:head 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:e960b815:::9:head 2024-02-05T07:56:35.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:52ea6a34:::10:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:89d3ae78:::11:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:de5d7c5f:::12:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:566253c9:::13:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:62a1935d:::14:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:863748b0:::15:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:3958e169:::16:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:4d4dabf9:::17:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:8391935d:::18:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:28883081:::19:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:69259c59:::20:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:4bdb80b7:::21:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:a11c5d71:::22:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:271af37b:::23:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:95b121be:::24:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:58d1031b:::25:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:0a050783:::26:head 2024-02-05T07:56:35.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:c709704c:::27:head 2024-02-05T07:56:35.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:cbe56eaf:::28:head 2024-02-05T07:56:35.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:86b4b162:::29:head 2024-02-05T07:56:35.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:70d89383:::30:head 2024-02-05T07:56:35.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:dd450c7c:::31:head 2024-02-05T07:56:35.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:6d5729b1:::32:head 2024-02-05T07:56:35.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:c388f3fb:::33:head 2024-02-05T07:56:35.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:56cfea31:::34:head 2024-02-05T07:56:35.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:9dbc1bf7:::35:head 2024-02-05T07:56:35.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:40b74ccd:::36:head 2024-02-05T07:56:35.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:4d5aaf42:::37:head 2024-02-05T07:56:35.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:920f362c:::38:head 2024-02-05T07:56:35.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:6cc53222:::39:head 2024-02-05T07:56:35.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:9cad833f:::40:head 2024-02-05T07:56:35.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:1ea84d41:::41:head 2024-02-05T07:56:35.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:c4480ef6:::42:head 2024-02-05T07:56:35.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:a694361e:::43:head 2024-02-05T07:56:35.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:d1bd33e9:::44:head 2024-02-05T07:56:35.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:ddc2cd5d:::45:head 2024-02-05T07:56:35.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:2b782207:::46:head 2024-02-05T07:56:35.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:7b187fca:::47:head 2024-02-05T07:56:35.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:90ecdf6f:::48:head 2024-02-05T07:56:35.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:a5ed95fe:::49:head 2024-02-05T07:56:35.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:ea0eaa55:::50:head 2024-02-05T07:56:35.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:f33ef17b:::51:head 2024-02-05T07:56:35.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:a0d1b2f6:::52:head 2024-02-05T07:56:35.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:60c5229e:::53:head 2024-02-05T07:56:35.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:edcbc575:::54:head 2024-02-05T07:56:35.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:102cf253:::55:head 2024-02-05T07:56:35.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:efb7fb0b:::56:head 2024-02-05T07:56:35.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:50d0a326:::57:head 2024-02-05T07:56:35.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:d4dc5daf:::58:head 2024-02-05T07:56:35.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:3a130462:::59:head 2024-02-05T07:56:35.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:ec87ed71:::60:head 2024-02-05T07:56:35.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:d5bc9454:::61:head 2024-02-05T07:56:35.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:3ddfe313:::62:head 2024-02-05T07:56:35.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:7c2816b9:::63:head 2024-02-05T07:56:35.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:47e00e4d:::64:head 2024-02-05T07:56:35.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:c6410c18:::65:head 2024-02-05T07:56:35.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:b48ed237:::66:head 2024-02-05T07:56:35.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:cd63ad31:::67:head 2024-02-05T07:56:35.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:b179e92b:::68:head 2024-02-05T07:56:35.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:0d9f741a:::69:head 2024-02-05T07:56:35.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:6d3352ae:::70:head 2024-02-05T07:56:35.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:c6d5c19e:::71:head 2024-02-05T07:56:35.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:bc4729c3:::72:head 2024-02-05T07:56:35.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:77e930b9:::73:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:0abeecfd:::74:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:b7c37e15:::75:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:b6378398:::76:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:02bd68de:::77:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:cc795d2d:::78:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:630d4fea:::79:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:e0d29ef5:::80:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:fd6f13d2:::81:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:606461d5:::82:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:eadbdc43:::83:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:8761d0bb:::84:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:9ef0186f:::85:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:e0d41294:::86:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:961de695:::87:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:1423148f:::88:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:633a8fa2:::89:head 2024-02-05T07:56:35.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:a8653809:::90:head 2024-02-05T07:56:35.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:3dac8b33:::91:head 2024-02-05T07:56:35.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:35aad435:::92:head 2024-02-05T07:56:35.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:f6dcc343:::93:head 2024-02-05T07:56:35.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:dbbdad87:::94:head 2024-02-05T07:56:35.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:1cb48ce0:::95:head 2024-02-05T07:56:35.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:03cd461c:::96:head 2024-02-05T07:56:35.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:17a4ea99:::97:head 2024-02-05T07:56:35.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:9993c9a7:::98:head 2024-02-05T07:56:35.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:6394211c:::99:head 2024-02-05T07:56:35.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:94c7ae57:::100:head 2024-02-05T07:56:35.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:6fdee5bb:::101:head 2024-02-05T07:56:35.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:9a477fd1:::102:head 2024-02-05T07:56:35.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:eb850916:::103:head 2024-02-05T07:56:35.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:affc56b9:::104:head 2024-02-05T07:56:35.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:b42dc814:::105:head 2024-02-05T07:56:35.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:f319f8f0:::106:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:9a40b9de:::107:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:8b524f28:::108:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:e3de589f:::109:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:90f90a5b:::110:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:a7b4f1d7:::111:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:af51766e:::112:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:b6f90bd1:::113:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:e0261208:::114:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:c9569ef7:::115:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:61bebe50:::116:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:fe93412b:::117:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:d3d38bee:::118:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:3100ba0c:::119:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:d0560ada:::120:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:f0ea8b35:::121:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:766f231a:::122:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:a07a2582:::123:head 2024-02-05T07:56:35.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:bd7c6b3a:::124:head 2024-02-05T07:56:35.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:fb2ddaff:::125:head 2024-02-05T07:56:35.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:4408e1fe:::126:head 2024-02-05T07:56:35.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:ee1df7a7:::127:head 2024-02-05T07:56:35.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:c3002909:::128:head 2024-02-05T07:56:35.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:4f48ffa9:::129:head 2024-02-05T07:56:35.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:edf38733:::130:head 2024-02-05T07:56:35.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:c08425c0:::131:head 2024-02-05T07:56:35.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:5f902d98:::132:head 2024-02-05T07:56:35.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:41ea2c93:::133:head 2024-02-05T07:56:35.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:813cee13:::134:head 2024-02-05T07:56:35.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:0131818d:::135:head 2024-02-05T07:56:35.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:26ba5a85:::136:head 2024-02-05T07:56:35.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:381b8a5a:::137:head 2024-02-05T07:56:35.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:28797e47:::138:head 2024-02-05T07:56:35.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:bfca7f22:::139:head 2024-02-05T07:56:35.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:36807075:::140:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:80b03975:::141:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:5c15709b:::142:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:f39ea15e:::143:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:ea992956:::144:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:48887b1c:::145:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:9f24a9dd:::146:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:987f100b:::147:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:d2dd3581:::148:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:7fed1808:::149:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:c80b70e9:::150:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:85ed90f9:::151:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:36428b24:::152:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:d044c34a:::153:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:7c18bf58:::154:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:d1c21232:::155:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:a7a3c575:::156:head 2024-02-05T07:56:35.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:87da0633:::157:head 2024-02-05T07:56:35.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:d5ac3822:::158:head 2024-02-05T07:56:35.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:3f20522d:::159:head 2024-02-05T07:56:35.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:6ca26563:::160:head 2024-02-05T07:56:35.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:532ce135:::161:head 2024-02-05T07:56:35.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:c78863e6:::162:head 2024-02-05T07:56:35.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:56:35.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-82060-57"}]': finished 2024-02-05T07:56:35.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:35 smithi157 ceph-mon[33345]: osdmap e304: 8 total, 8 up, 8 in 2024-02-05T07:56:35.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-05T07:56:35.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-05T07:56:35.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:35.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:35.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-05T07:56:35.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-82060-57"}]': finished 2024-02-05T07:56:35.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:35 smithi157 ceph-mon[33345]: osdmap e305: 8 total, 8 up, 8 in 2024-02-05T07:56:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:56:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-82060-57"}]': finished 2024-02-05T07:56:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[41654]: osdmap e304: 8 total, 8 up, 8 in 2024-02-05T07:56:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-05T07:56:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-05T07:56:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-05T07:56:36.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-82060-57"}]': finished 2024-02-05T07:56:36.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[41654]: osdmap e305: 8 total, 8 up, 8 in 2024-02-05T07:56:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:56:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-82060-57"}]': finished 2024-02-05T07:56:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[29507]: osdmap e304: 8 total, 8 up, 8 in 2024-02-05T07:56:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-05T07:56:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-05T07:56:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3961873914' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-82060-57"}]: dispatch 2024-02-05T07:56:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-05T07:56:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-82060-57"}]': finished 2024-02-05T07:56:36.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:35 smithi131 ceph-mon[29507]: osdmap e305: 8 total, 8 up, 8 in 2024-02-05T07:56:36.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:36 smithi157 ceph-mon[33345]: pgmap v418: 292 pgs: 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:36.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:36.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:36.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:36.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:36.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-82060-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:36.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-82060-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:36.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi131-82290-43","var": "pg_num","format": "json"}]: dispatch 2024-02-05T07:56:36.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:36.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:36.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43"}]: dispatch 2024-02-05T07:56:36.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43"}]: dispatch 2024-02-05T07:56:37.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[29507]: pgmap v418: 292 pgs: 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:37.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:37.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:37.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:37.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:37.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-82060-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:37.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-82060-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:37.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi131-82290-43","var": "pg_num","format": "json"}]: dispatch 2024-02-05T07:56:37.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:37.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:37.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43"}]: dispatch 2024-02-05T07:56:37.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43"}]: dispatch 2024-02-05T07:56:37.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[41654]: pgmap v418: 292 pgs: 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:37.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:37.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:37.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:37.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:37.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-82060-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:37.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-82060-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:37.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi131-82290-43","var": "pg_num","format": "json"}]: dispatch 2024-02-05T07:56:37.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:37.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:37.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43"}]: dispatch 2024-02-05T07:56:37.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43"}]: dispatch 2024-02-05T07:56:37.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-82060-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:37.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43"}]': finished 2024-02-05T07:56:37.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:37 smithi157 ceph-mon[33345]: osdmap e306: 8 total, 8 up, 8 in 2024-02-05T07:56:37.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-82060-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:37.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-82060-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:38.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-82060-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:38.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43"}]': finished 2024-02-05T07:56:38.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:37 smithi131 ceph-mon[41654]: osdmap e306: 8 total, 8 up, 8 in 2024-02-05T07:56:38.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-82060-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:38.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-82060-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:38.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-82060-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:38.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-43"}]': finished 2024-02-05T07:56:38.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:37 smithi131 ceph-mon[29507]: osdmap e306: 8 total, 8 up, 8 in 2024-02-05T07:56:38.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-82060-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:38.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-82060-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:38.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:56:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:56:38.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:38 smithi157 ceph-mon[33345]: pgmap v421: 292 pgs: 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:38.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:38 smithi157 ceph-mon[33345]: osdmap e307: 8 total, 8 up, 8 in 2024-02-05T07:56:39.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:38 smithi131 ceph-mon[41654]: pgmap v421: 292 pgs: 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:39.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:38 smithi131 ceph-mon[41654]: osdmap e307: 8 total, 8 up, 8 in 2024-02-05T07:56:39.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:38 smithi131 ceph-mon[29507]: pgmap v421: 292 pgs: 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:39.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:38 smithi131 ceph-mon[29507]: osdmap e307: 8 total, 8 up, 8 in 2024-02-05T07:56:39.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:39 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:39.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-82060-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-82060-58"}]': finished 2024-02-05T07:56:39.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:39 smithi157 ceph-mon[33345]: osdmap e308: 8 total, 8 up, 8 in 2024-02-05T07:56:39.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:39 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:39.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:40.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:39 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:40.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-82060-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-82060-58"}]': finished 2024-02-05T07:56:40.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:39 smithi131 ceph-mon[41654]: osdmap e308: 8 total, 8 up, 8 in 2024-02-05T07:56:40.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:39 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:40.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:40.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:39 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:40.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-82060-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-82060-58"}]': finished 2024-02-05T07:56:40.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:39 smithi131 ceph-mon[29507]: osdmap e308: 8 total, 8 up, 8 in 2024-02-05T07:56:40.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:40.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:40.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:40 smithi157 ceph-mon[33345]: pgmap v424: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:40.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:40 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:40.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:40.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:40 smithi157 ceph-mon[33345]: osdmap e309: 8 total, 8 up, 8 in 2024-02-05T07:56:40.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:40 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:40.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:41.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:40 smithi131 ceph-mon[41654]: pgmap v424: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:41.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:40 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:41.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:41.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:40 smithi131 ceph-mon[41654]: osdmap e309: 8 total, 8 up, 8 in 2024-02-05T07:56:41.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:40 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:41.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:41.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:40 smithi131 ceph-mon[29507]: pgmap v424: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:41.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:40 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:41.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:41.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:40 smithi131 ceph-mon[29507]: osdmap e309: 8 total, 8 up, 8 in 2024-02-05T07:56:41.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:40 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:41.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:56:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:56:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:56:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:41 smithi157 ceph-mon[33345]: osdmap e310: 8 total, 8 up, 8 in 2024-02-05T07:56:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:56:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:56:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:56:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:41 smithi131 ceph-mon[29507]: osdmap e310: 8 total, 8 up, 8 in 2024-02-05T07:56:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:56:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:42.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:56:42.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:56:42.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:41 smithi131 ceph-mon[41654]: osdmap e310: 8 total, 8 up, 8 in 2024-02-05T07:56:42.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:56:42.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:42.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:42 smithi157 ceph-mon[33345]: pgmap v427: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_count","val": "3"}]': finished 2024-02-05T07:56:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-82060-58"}]': finished 2024-02-05T07:56:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:42 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:56:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:42 smithi157 ceph-mon[33345]: osdmap e311: 8 total, 8 up, 8 in 2024-02-05T07:56:42.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:56:42.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:42 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:42.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:43.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:56:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:56:43.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-mon[41654]: pgmap v427: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:43.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_count","val": "3"}]': finished 2024-02-05T07:56:43.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-82060-58"}]': finished 2024-02-05T07:56:43.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:56:43.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-mon[41654]: osdmap e311: 8 total, 8 up, 8 in 2024-02-05T07:56:43.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:56:43.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:43.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:43.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-mon[29507]: pgmap v427: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:43.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_count","val": "3"}]': finished 2024-02-05T07:56:43.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-82060-58"}]': finished 2024-02-05T07:56:43.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:56:43.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-mon[29507]: osdmap e311: 8 total, 8 up, 8 in 2024-02-05T07:56:43.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:56:43.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3066771453' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:43.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-82060-58"}]: dispatch 2024-02-05T07:56:43.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_period","val": "3"}]': finished 2024-02-05T07:56:43.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-82060-58"}]': finished 2024-02-05T07:56:43.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:56:43.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:43 smithi157 ceph-mon[33345]: osdmap e312: 8 total, 8 up, 8 in 2024-02-05T07:56:43.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:56:43.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-82060-59"}]: dispatch 2024-02-05T07:56:43.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-82060-59"}]: dispatch 2024-02-05T07:56:43.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi131-82060-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:43.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:56:43.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi131-82060-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:43.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:43 smithi157 ceph-mon[33345]: osdmap e313: 8 total, 8 up, 8 in 2024-02-05T07:56:43.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:56:43.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi131-82060-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi131-82060-59"}]: dispatch 2024-02-05T07:56:43.973 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:56:44.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_period","val": "3"}]': finished 2024-02-05T07:56:44.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-82060-58"}]': finished 2024-02-05T07:56:44.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:56:44.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[29507]: osdmap e312: 8 total, 8 up, 8 in 2024-02-05T07:56:44.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:56:44.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-82060-59"}]: dispatch 2024-02-05T07:56:44.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-82060-59"}]: dispatch 2024-02-05T07:56:44.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi131-82060-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:44.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:56:44.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi131-82060-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:44.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[29507]: osdmap e313: 8 total, 8 up, 8 in 2024-02-05T07:56:44.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:56:44.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi131-82060-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi131-82060-59"}]: dispatch 2024-02-05T07:56:44.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:56:44.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_period","val": "3"}]': finished 2024-02-05T07:56:44.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-82060-58"}]': finished 2024-02-05T07:56:44.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:56:44.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[41654]: osdmap e312: 8 total, 8 up, 8 in 2024-02-05T07:56:44.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:56:44.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-82060-59"}]: dispatch 2024-02-05T07:56:44.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-82060-59"}]: dispatch 2024-02-05T07:56:44.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi131-82060-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:44.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:56:44.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi131-82060-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:44.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[41654]: osdmap e313: 8 total, 8 up, 8 in 2024-02-05T07:56:44.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:56:44.020 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi131-82060-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi131-82060-59"}]: dispatch 2024-02-05T07:56:44.021 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:56:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:44 smithi157 ceph-mon[33345]: pgmap v430: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:44 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-05T07:56:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:44 smithi131 ceph-mon[41654]: pgmap v430: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:44 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-05T07:56:45.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:44 smithi131 ceph-mon[29507]: pgmap v430: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:45.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:44 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:45.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-05T07:56:46.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:45 smithi157 ceph-mon[33345]: osdmap e314: 8 total, 8 up, 8 in 2024-02-05T07:56:46.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi131-82060-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi131-82060-59"}]': finished 2024-02-05T07:56:46.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:45 smithi157 ceph-mon[33345]: osdmap e315: 8 total, 8 up, 8 in 2024-02-05T07:56:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:45 smithi131 ceph-mon[41654]: osdmap e314: 8 total, 8 up, 8 in 2024-02-05T07:56:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi131-82060-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi131-82060-59"}]': finished 2024-02-05T07:56:46.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:45 smithi131 ceph-mon[41654]: osdmap e315: 8 total, 8 up, 8 in 2024-02-05T07:56:46.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:45 smithi131 ceph-mon[29507]: osdmap e314: 8 total, 8 up, 8 in 2024-02-05T07:56:46.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi131-82060-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi131-82060-59"}]': finished 2024-02-05T07:56:46.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:45 smithi131 ceph-mon[29507]: osdmap e315: 8 total, 8 up, 8 in 2024-02-05T07:56:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:46 smithi157 ceph-mon[33345]: pgmap v433: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:46 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:46 smithi157 ceph-mon[33345]: osdmap e316: 8 total, 8 up, 8 in 2024-02-05T07:56:47.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:46 smithi131 ceph-mon[41654]: pgmap v433: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:47.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:46 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:47.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:46 smithi131 ceph-mon[41654]: osdmap e316: 8 total, 8 up, 8 in 2024-02-05T07:56:47.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:46 smithi131 ceph-mon[29507]: pgmap v433: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:47.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:46 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:47.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:46 smithi131 ceph-mon[29507]: osdmap e316: 8 total, 8 up, 8 in 2024-02-05T07:56:48.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:56:48 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:56:48.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:48 smithi157 ceph-mon[33345]: pgmap v436: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:48.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:48 smithi157 ceph-mon[33345]: osdmap e317: 8 total, 8 up, 8 in 2024-02-05T07:56:48.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-82060-59"}]: dispatch 2024-02-05T07:56:49.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:48 smithi131 ceph-mon[41654]: pgmap v436: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:49.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:48 smithi131 ceph-mon[41654]: osdmap e317: 8 total, 8 up, 8 in 2024-02-05T07:56:49.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-82060-59"}]: dispatch 2024-02-05T07:56:49.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:48 smithi131 ceph-mon[29507]: pgmap v436: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:49.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:48 smithi131 ceph-mon[29507]: osdmap e317: 8 total, 8 up, 8 in 2024-02-05T07:56:49.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-82060-59"}]: dispatch 2024-02-05T07:56:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:49 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-82060-59"}]': finished 2024-02-05T07:56:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:49 smithi131 ceph-mon[41654]: osdmap e318: 8 total, 8 up, 8 in 2024-02-05T07:56:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-82060-59"}]: dispatch 2024-02-05T07:56:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-82060-59"}]': finished 2024-02-05T07:56:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:49 smithi131 ceph-mon[41654]: osdmap e319: 8 total, 8 up, 8 in 2024-02-05T07:56:50.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:49 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:50.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-82060-59"}]': finished 2024-02-05T07:56:50.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:49 smithi131 ceph-mon[29507]: osdmap e318: 8 total, 8 up, 8 in 2024-02-05T07:56:50.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-82060-59"}]: dispatch 2024-02-05T07:56:50.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-82060-59"}]': finished 2024-02-05T07:56:50.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:49 smithi131 ceph-mon[29507]: osdmap e319: 8 total, 8 up, 8 in 2024-02-05T07:56:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:49 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-82060-59"}]': finished 2024-02-05T07:56:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:49 smithi157 ceph-mon[33345]: osdmap e318: 8 total, 8 up, 8 in 2024-02-05T07:56:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-82060-59"}]: dispatch 2024-02-05T07:56:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/108189067' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-82060-59"}]': finished 2024-02-05T07:56:50.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:49 smithi157 ceph-mon[33345]: osdmap e319: 8 total, 8 up, 8 in 2024-02-05T07:56:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:50 smithi131 ceph-mon[41654]: pgmap v439: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:56:51.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:50 smithi131 ceph-mon[29507]: pgmap v439: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:56:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:50 smithi157 ceph-mon[33345]: pgmap v439: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:56:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:51 smithi131 ceph-mon[41654]: osdmap e320: 8 total, 8 up, 8 in 2024-02-05T07:56:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3025102950' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-82060-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:52.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:51 smithi131 ceph-mon[29507]: osdmap e320: 8 total, 8 up, 8 in 2024-02-05T07:56:52.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3025102950' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-82060-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:52.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:51 smithi157 ceph-mon[33345]: osdmap e320: 8 total, 8 up, 8 in 2024-02-05T07:56:52.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3025102950' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-82060-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:56:52.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:7462ddf6:::.RoundTripAppendPP (3252 ms) 2024-02-05T07:56:52.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-02-05T07:56:52.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (2762 ms) 2024-02-05T07:56:52.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-02-05T07:56:52.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3234 ms) 2024-02-05T07:56:52.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-02-05T07:56:52.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (2766 ms) 2024-02-05T07:56:52.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-02-05T07:56:52.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: setting pool EIO 2024-02-05T07:56:52.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: max_success 99, min_failed 100 2024-02-05T07:56:52.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4007 ms) 2024-02-05T07:56:52.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-02-05T07:56:52.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3006 ms) 2024-02-05T07:56:52.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (123846 ms total) 2024-02-05T07:56:52.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: 2024-02-05T07:56:52.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-02-05T07:56:52.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-02-05T07:56:52.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3008 ms) 2024-02-05T07:56:52.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-02-05T07:56:52.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9267 ms) 2024-02-05T07:56:52.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-02-05T07:56:52.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14706 ms) 2024-02-05T07:56:52.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-02-05T07:56:52.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3011 ms) 2024-02-05T07:56:52.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29993 ms total) 2024-02-05T07:56:52.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: 2024-02-05T07:56:52.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-02-05T07:56:52.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-05T07:56:52.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3251 ms) 2024-02-05T07:56:52.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3251 ms total) 2024-02-05T07:56:52.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: 2024-02-05T07:56:52.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-02-05T07:56:52.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-02-05T07:56:52.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13745 ms) 2024-02-05T07:56:52.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-02-05T07:56:52.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (7001 ms) 2024-02-05T07:56:52.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-02-05T07:56:52.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7013 ms) 2024-02-05T07:56:52.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-02-05T07:56:52.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7039 ms) 2024-02-05T07:56:52.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-02-05T07:56:52.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3018 ms) 2024-02-05T07:56:52.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-02-05T07:56:52.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7021 ms) 2024-02-05T07:56:52.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-02-05T07:56:52.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6925 ms) 2024-02-05T07:56:52.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-02-05T07:56:52.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7022 ms) 2024-02-05T07:56:52.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-02-05T07:56:52.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7020 ms) 2024-02-05T07:56:52.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-02-05T07:56:52.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7010 ms) 2024-02-05T07:56:52.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-02-05T07:56:52.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7030 ms) 2024-02-05T07:56:52.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-02-05T07:56:52.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7061 ms) 2024-02-05T07:56:52.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-02-05T07:56:52.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7013 ms) 2024-02-05T07:56:52.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-02-05T07:56:53.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:56:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:56:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:56:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:52 smithi131 ceph-mon[29507]: pgmap v442: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:56:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:52 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3025102950' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-82060-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:52 smithi131 ceph-mon[29507]: osdmap e321: 8 total, 8 up, 8 in 2024-02-05T07:56:53.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:52 smithi131 ceph-mon[41654]: pgmap v442: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:56:53.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:52 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:53.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3025102950' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-82060-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:53.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:52 smithi131 ceph-mon[41654]: osdmap e321: 8 total, 8 up, 8 in 2024-02-05T07:56:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:52 smithi157 ceph-mon[33345]: pgmap v442: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:56:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:52 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3025102950' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-82060-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:56:53.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:52 smithi157 ceph-mon[33345]: osdmap e321: 8 total, 8 up, 8 in 2024-02-05T07:56:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:53 smithi131 ceph-mon[41654]: osdmap e322: 8 total, 8 up, 8 in 2024-02-05T07:56:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-82060-61"}]: dispatch 2024-02-05T07:56:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-82060-61"}]: dispatch 2024-02-05T07:56:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi131-82060-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:54.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:53 smithi131 ceph-mon[29507]: osdmap e322: 8 total, 8 up, 8 in 2024-02-05T07:56:54.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-82060-61"}]: dispatch 2024-02-05T07:56:54.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-82060-61"}]: dispatch 2024-02-05T07:56:54.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi131-82060-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:54.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:53 smithi157 ceph-mon[33345]: osdmap e322: 8 total, 8 up, 8 in 2024-02-05T07:56:54.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-82060-61"}]: dispatch 2024-02-05T07:56:54.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-82060-61"}]: dispatch 2024-02-05T07:56:54.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi131-82060-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:56:55.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:55 smithi131 ceph-mon[41654]: pgmap v445: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:56:55.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi131-82060-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:55.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:55 smithi131 ceph-mon[41654]: osdmap e323: 8 total, 8 up, 8 in 2024-02-05T07:56:55.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi131-82060-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi131-82060-61"}]: dispatch 2024-02-05T07:56:55.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:55 smithi131 ceph-mon[29507]: pgmap v445: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:56:55.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi131-82060-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:55.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:55 smithi131 ceph-mon[29507]: osdmap e323: 8 total, 8 up, 8 in 2024-02-05T07:56:55.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi131-82060-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi131-82060-61"}]: dispatch 2024-02-05T07:56:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:55 smithi157 ceph-mon[33345]: pgmap v445: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:56:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi131-82060-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:56:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:55 smithi157 ceph-mon[33345]: osdmap e323: 8 total, 8 up, 8 in 2024-02-05T07:56:55.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi131-82060-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi131-82060-61"}]: dispatch 2024-02-05T07:56:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:56 smithi131 ceph-mon[41654]: osdmap e324: 8 total, 8 up, 8 in 2024-02-05T07:56:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi131-82060-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi131-82060-61"}]': finished 2024-02-05T07:56:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:56 smithi131 ceph-mon[41654]: osdmap e325: 8 total, 8 up, 8 in 2024-02-05T07:56:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:56 smithi131 ceph-mon[29507]: osdmap e324: 8 total, 8 up, 8 in 2024-02-05T07:56:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi131-82060-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi131-82060-61"}]': finished 2024-02-05T07:56:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:56 smithi131 ceph-mon[29507]: osdmap e325: 8 total, 8 up, 8 in 2024-02-05T07:56:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:56 smithi157 ceph-mon[33345]: osdmap e324: 8 total, 8 up, 8 in 2024-02-05T07:56:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi131-82060-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi131-82060-61"}]': finished 2024-02-05T07:56:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:56 smithi157 ceph-mon[33345]: osdmap e325: 8 total, 8 up, 8 in 2024-02-05T07:56:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:57 smithi131 ceph-mon[29507]: pgmap v448: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:56:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:57 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:57 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:57 smithi131 ceph-mon[29507]: osdmap e326: 8 total, 8 up, 8 in 2024-02-05T07:56:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:57 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45"}]: dispatch 2024-02-05T07:56:57.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:57 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45"}]: dispatch 2024-02-05T07:56:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:57 smithi131 ceph-mon[41654]: pgmap v448: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:56:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:57 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:57 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:57 smithi131 ceph-mon[41654]: osdmap e326: 8 total, 8 up, 8 in 2024-02-05T07:56:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:57 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45"}]: dispatch 2024-02-05T07:56:57.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:57 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45"}]: dispatch 2024-02-05T07:56:57.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:57 smithi157 ceph-mon[33345]: pgmap v448: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:56:57.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:57 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:57.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:57 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:56:57.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:57 smithi157 ceph-mon[33345]: osdmap e326: 8 total, 8 up, 8 in 2024-02-05T07:56:57.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:57 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45"}]: dispatch 2024-02-05T07:56:57.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:57 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45"}]: dispatch 2024-02-05T07:56:58.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:56:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:56:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:58 smithi157 ceph-mon[33345]: pgmap v451: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45"}]': finished 2024-02-05T07:56:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:58 smithi157 ceph-mon[33345]: osdmap e327: 8 total, 8 up, 8 in 2024-02-05T07:56:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-82060-61"}]: dispatch 2024-02-05T07:56:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:58 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:58 smithi131 ceph-mon[41654]: pgmap v451: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45"}]': finished 2024-02-05T07:56:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:58 smithi131 ceph-mon[41654]: osdmap e327: 8 total, 8 up, 8 in 2024-02-05T07:56:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-82060-61"}]: dispatch 2024-02-05T07:56:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:58 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:56:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:58 smithi131 ceph-mon[29507]: pgmap v451: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:56:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-45"}]': finished 2024-02-05T07:56:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:58 smithi131 ceph-mon[29507]: osdmap e327: 8 total, 8 up, 8 in 2024-02-05T07:56:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-82060-61"}]: dispatch 2024-02-05T07:56:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:58 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:59 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-82060-61"}]': finished 2024-02-05T07:57:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:59 smithi157 ceph-mon[33345]: osdmap e328: 8 total, 8 up, 8 in 2024-02-05T07:57:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-82060-61"}]: dispatch 2024-02-05T07:57:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:59 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:56:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-82060-61"}]': finished 2024-02-05T07:57:00.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:59 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:00.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-82060-61"}]': finished 2024-02-05T07:57:00.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:59 smithi131 ceph-mon[41654]: osdmap e328: 8 total, 8 up, 8 in 2024-02-05T07:57:00.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-82060-61"}]: dispatch 2024-02-05T07:57:00.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:59 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:00.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:56:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-82060-61"}]': finished 2024-02-05T07:57:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:59 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-82060-61"}]': finished 2024-02-05T07:57:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:59 smithi131 ceph-mon[29507]: osdmap e328: 8 total, 8 up, 8 in 2024-02-05T07:57:00.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-82060-61"}]: dispatch 2024-02-05T07:57:00.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:59 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:00.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:56:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/379998660' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-82060-61"}]': finished 2024-02-05T07:57:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:00 smithi157 ceph-mon[33345]: pgmap v454: 260 pgs: 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:00 smithi157 ceph-mon[33345]: osdmap e329: 8 total, 8 up, 8 in 2024-02-05T07:57:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:01.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-82060-62"}]: dispatch 2024-02-05T07:57:01.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-82060-62"}]: dispatch 2024-02-05T07:57:01.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi131-82060-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:01.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:57:01.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi131-82060-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:57:01.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:00 smithi157 ceph-mon[33345]: osdmap e330: 8 total, 8 up, 8 in 2024-02-05T07:57:01.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi131-82060-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi131-82060-62"}]: dispatch 2024-02-05T07:57:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[41654]: pgmap v454: 260 pgs: 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[41654]: osdmap e329: 8 total, 8 up, 8 in 2024-02-05T07:57:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-82060-62"}]: dispatch 2024-02-05T07:57:01.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-82060-62"}]: dispatch 2024-02-05T07:57:01.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi131-82060-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:01.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:57:01.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi131-82060-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:57:01.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[41654]: osdmap e330: 8 total, 8 up, 8 in 2024-02-05T07:57:01.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi131-82060-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi131-82060-62"}]: dispatch 2024-02-05T07:57:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[29507]: pgmap v454: 260 pgs: 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[29507]: osdmap e329: 8 total, 8 up, 8 in 2024-02-05T07:57:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-82060-62"}]: dispatch 2024-02-05T07:57:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-82060-62"}]: dispatch 2024-02-05T07:57:01.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi131-82060-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:01.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:57:01.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi131-82060-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:57:01.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[29507]: osdmap e330: 8 total, 8 up, 8 in 2024-02-05T07:57:01.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi131-82060-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi131-82060-62"}]: dispatch 2024-02-05T07:57:02.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:02.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:02.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:57:02.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:02.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:02.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:57:02.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:02.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:02.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:57:03.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:57:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:57:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:57:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[29507]: pgmap v457: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[29507]: osdmap e331: 8 total, 8 up, 8 in 2024-02-05T07:57:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi131-82060-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi131-82060-62"}]': finished 2024-02-05T07:57:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-47"}]': finished 2024-02-05T07:57:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[29507]: osdmap e332: 8 total, 8 up, 8 in 2024-02-05T07:57:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-47", "mode": "writeback"}]: dispatch 2024-02-05T07:57:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-47", "mode": "writeback"}]: dispatch 2024-02-05T07:57:03.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[41654]: pgmap v457: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:03.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:03.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[41654]: osdmap e331: 8 total, 8 up, 8 in 2024-02-05T07:57:03.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:03.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi131-82060-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi131-82060-62"}]': finished 2024-02-05T07:57:03.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-47"}]': finished 2024-02-05T07:57:03.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[41654]: osdmap e332: 8 total, 8 up, 8 in 2024-02-05T07:57:03.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-47", "mode": "writeback"}]: dispatch 2024-02-05T07:57:03.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-47", "mode": "writeback"}]: dispatch 2024-02-05T07:57:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:03 smithi157 ceph-mon[33345]: pgmap v457: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:03 smithi157 ceph-mon[33345]: osdmap e331: 8 total, 8 up, 8 in 2024-02-05T07:57:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi131-82060-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi131-82060-62"}]': finished 2024-02-05T07:57:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-47"}]': finished 2024-02-05T07:57:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:03 smithi157 ceph-mon[33345]: osdmap e332: 8 total, 8 up, 8 in 2024-02-05T07:57:03.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-47", "mode": "writeback"}]: dispatch 2024-02-05T07:57:03.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-47", "mode": "writeback"}]: dispatch 2024-02-05T07:57:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:04 smithi157 ceph-mon[33345]: pgmap v460: 300 pgs: 8 creating+peering, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:57:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:04 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:57:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-47", "mode": "writeback"}]': finished 2024-02-05T07:57:04.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:04 smithi157 ceph-mon[33345]: osdmap e333: 8 total, 8 up, 8 in 2024-02-05T07:57:04.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:57:04.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:57:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:04 smithi131 ceph-mon[41654]: pgmap v460: 300 pgs: 8 creating+peering, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:57:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:04 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:57:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-47", "mode": "writeback"}]': finished 2024-02-05T07:57:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:04 smithi131 ceph-mon[41654]: osdmap e333: 8 total, 8 up, 8 in 2024-02-05T07:57:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:57:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:57:04.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:04 smithi131 ceph-mon[29507]: pgmap v460: 300 pgs: 8 creating+peering, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:57:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:04 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:57:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-47", "mode": "writeback"}]': finished 2024-02-05T07:57:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:04 smithi131 ceph-mon[29507]: osdmap e333: 8 total, 8 up, 8 in 2024-02-05T07:57:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:57:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:57:05.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:05 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:05.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:57:05.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:57:05.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:05 smithi157 ceph-mon[33345]: osdmap e334: 8 total, 8 up, 8 in 2024-02-05T07:57:05.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:57:05.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-82060-62"}]: dispatch 2024-02-05T07:57:05.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:05 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:05.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:57:05.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:57:05.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:05 smithi131 ceph-mon[41654]: osdmap e334: 8 total, 8 up, 8 in 2024-02-05T07:57:05.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:57:05.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-82060-62"}]: dispatch 2024-02-05T07:57:05.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:05 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:05.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:57:05.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:57:05.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:05 smithi131 ceph-mon[29507]: osdmap e334: 8 total, 8 up, 8 in 2024-02-05T07:57:05.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:57:05.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-82060-62"}]: dispatch 2024-02-05T07:57:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:06 smithi157 ceph-mon[33345]: pgmap v463: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:57:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:57:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-82060-62"}]': finished 2024-02-05T07:57:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:57:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:06 smithi157 ceph-mon[33345]: osdmap e335: 8 total, 8 up, 8 in 2024-02-05T07:57:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-82060-62"}]: dispatch 2024-02-05T07:57:07.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:57:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:06 smithi131 ceph-mon[41654]: pgmap v463: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:57:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:57:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-82060-62"}]': finished 2024-02-05T07:57:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:57:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:06 smithi131 ceph-mon[41654]: osdmap e335: 8 total, 8 up, 8 in 2024-02-05T07:57:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-82060-62"}]: dispatch 2024-02-05T07:57:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:57:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:06 smithi131 ceph-mon[29507]: pgmap v463: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:57:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:57:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-82060-62"}]': finished 2024-02-05T07:57:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:57:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:06 smithi131 ceph-mon[29507]: osdmap e335: 8 total, 8 up, 8 in 2024-02-05T07:57:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-82060-62"}]: dispatch 2024-02-05T07:57:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:57:08.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:07 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:57:08.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:07 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-82060-62"}]': finished 2024-02-05T07:57:08.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:57:08.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:07 smithi157 ceph-mon[33345]: osdmap e336: 8 total, 8 up, 8 in 2024-02-05T07:57:08.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:07 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:57:08.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:57:08.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:07 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:08.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:08.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:07 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:08.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:08.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:07 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-82060-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:08.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-82060-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:08.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:57:08.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-82060-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:57:08.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:07 smithi157 ceph-mon[33345]: osdmap e337: 8 total, 8 up, 8 in 2024-02-05T07:57:08.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:57:08.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-82060-62"}]': finished 2024-02-05T07:57:08.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:57:08.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[41654]: osdmap e336: 8 total, 8 up, 8 in 2024-02-05T07:57:08.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:57:08.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:57:08.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:08.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:08.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:08.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:08.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-82060-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:08.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-82060-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:08.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:57:08.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-82060-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:57:08.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[41654]: osdmap e337: 8 total, 8 up, 8 in 2024-02-05T07:57:08.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:57:08.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/654056354' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-82060-62"}]': finished 2024-02-05T07:57:08.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:57:08.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[29507]: osdmap e336: 8 total, 8 up, 8 in 2024-02-05T07:57:08.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:57:08.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:57:08.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:08.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:08.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:08.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:08.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-82060-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:08.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-82060-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:08.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:57:08.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-82060-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:57:08.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:07 smithi131 ceph-mon[29507]: osdmap e337: 8 total, 8 up, 8 in 2024-02-05T07:57:08.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:57:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:57:09.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:09 smithi157 ceph-mon[33345]: pgmap v466: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:57:09.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:57:09.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-82060-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:09.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:57:09.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-82060-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:09.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-05T07:57:09.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:09 smithi131 ceph-mon[41654]: pgmap v466: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:57:09.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:57:09.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-82060-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:09.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:57:09.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-82060-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:09.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-05T07:57:09.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:09 smithi131 ceph-mon[29507]: pgmap v466: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:57:09.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:57:09.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-82060-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:09.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:57:09.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-82060-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:09.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-05T07:57:10.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:10 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:10.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:57:10.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:10 smithi157 ceph-mon[33345]: osdmap e338: 8 total, 8 up, 8 in 2024-02-05T07:57:10.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:57:10.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:10 smithi157 ceph-mon[33345]: pgmap v469: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:10.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-82060-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-82060-63"}]': finished 2024-02-05T07:57:10.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-05T07:57:10.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:10 smithi157 ceph-mon[33345]: osdmap e339: 8 total, 8 up, 8 in 2024-02-05T07:57:10.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:10.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:57:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[41654]: osdmap e338: 8 total, 8 up, 8 in 2024-02-05T07:57:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:57:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[41654]: pgmap v469: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-82060-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-82060-63"}]': finished 2024-02-05T07:57:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-05T07:57:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[41654]: osdmap e339: 8 total, 8 up, 8 in 2024-02-05T07:57:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:57:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[29507]: osdmap e338: 8 total, 8 up, 8 in 2024-02-05T07:57:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:57:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[29507]: pgmap v469: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-82060-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-82060-63"}]': finished 2024-02-05T07:57:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-05T07:57:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[29507]: osdmap e339: 8 total, 8 up, 8 in 2024-02-05T07:57:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:10.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:57:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:11 smithi157 ceph-mon[33345]: osdmap e340: 8 total, 8 up, 8 in 2024-02-05T07:57:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:11 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:57:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:11 smithi131 ceph-mon[41654]: osdmap e340: 8 total, 8 up, 8 in 2024-02-05T07:57:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:11 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:12.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:57:12.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:11 smithi131 ceph-mon[29507]: osdmap e340: 8 total, 8 up, 8 in 2024-02-05T07:57:12.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:12.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:12.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:11 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:12 smithi157 ceph-mon[33345]: pgmap v472: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47"}]': finished 2024-02-05T07:57:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:12 smithi157 ceph-mon[33345]: osdmap e341: 8 total, 8 up, 8 in 2024-02-05T07:57:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:12 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:57:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:12 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:57:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:12 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:57:13.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:12 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:57:13.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:57:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:57:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[41654]: pgmap v472: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47"}]': finished 2024-02-05T07:57:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[41654]: osdmap e341: 8 total, 8 up, 8 in 2024-02-05T07:57:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:13.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:13.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:13.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:57:13.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:57:13.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:57:13.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:57:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[29507]: pgmap v472: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47"}]': finished 2024-02-05T07:57:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[29507]: osdmap e341: 8 total, 8 up, 8 in 2024-02-05T07:57:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:13.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:13.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:13.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-47"}]: dispatch 2024-02-05T07:57:13.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:57:13.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:57:13.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:57:13.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:12 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:57:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:13 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-82060-63"}]': finished 2024-02-05T07:57:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:13 smithi157 ceph-mon[33345]: osdmap e342: 8 total, 8 up, 8 in 2024-02-05T07:57:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:13 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:13 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:14.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:13 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-82060-63"}]': finished 2024-02-05T07:57:14.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:13 smithi131 ceph-mon[41654]: osdmap e342: 8 total, 8 up, 8 in 2024-02-05T07:57:14.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:13 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:14.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:13 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:14.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:13 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-82060-63"}]': finished 2024-02-05T07:57:14.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:13 smithi131 ceph-mon[29507]: osdmap e342: 8 total, 8 up, 8 in 2024-02-05T07:57:14.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:13 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3802768994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:14.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:13 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-82060-63"}]: dispatch 2024-02-05T07:57:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:14 smithi157 ceph-mon[33345]: pgmap v475: 260 pgs: 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:57:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-82060-63"}]': finished 2024-02-05T07:57:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:14 smithi157 ceph-mon[33345]: osdmap e343: 8 total, 8 up, 8 in 2024-02-05T07:57:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-82060-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-82060-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:15.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:14 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[41654]: pgmap v475: 260 pgs: 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:57:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-82060-63"}]': finished 2024-02-05T07:57:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[41654]: osdmap e343: 8 total, 8 up, 8 in 2024-02-05T07:57:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:15.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:15.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-82060-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:15.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-82060-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:15.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:15.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[29507]: pgmap v475: 260 pgs: 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:57:15.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-82060-63"}]': finished 2024-02-05T07:57:15.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[29507]: osdmap e343: 8 total, 8 up, 8 in 2024-02-05T07:57:15.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:15.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:15.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:15.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:15.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:15.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:15.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-82060-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:15.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-82060-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:15.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:14 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:16.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:57:16.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-82060-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:57:16.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:16 smithi157 ceph-mon[33345]: osdmap e344: 8 total, 8 up, 8 in 2024-02-05T07:57:16.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-82060-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:16.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-82060-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:16.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:16.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:16.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:57:16.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:57:16.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-82060-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:57:16.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:16 smithi131 ceph-mon[41654]: osdmap e344: 8 total, 8 up, 8 in 2024-02-05T07:57:16.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-82060-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:16.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-82060-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:16.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:16.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:16.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:57:16.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:57:16.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-82060-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:57:16.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:16 smithi131 ceph-mon[29507]: osdmap e344: 8 total, 8 up, 8 in 2024-02-05T07:57:16.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-82060-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:16.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-82060-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:16.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:16.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:16.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:57:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:17 smithi157 ceph-mon[33345]: pgmap v478: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:57:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:17 smithi157 ceph-mon[33345]: osdmap e345: 8 total, 8 up, 8 in 2024-02-05T07:57:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:17.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-82060-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-82060-64"}]': finished 2024-02-05T07:57:17.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-49"}]': finished 2024-02-05T07:57:17.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:17 smithi157 ceph-mon[33345]: osdmap e346: 8 total, 8 up, 8 in 2024-02-05T07:57:17.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-49", "mode": "readproxy"}]: dispatch 2024-02-05T07:57:17.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-49", "mode": "readproxy"}]: dispatch 2024-02-05T07:57:17.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[29507]: pgmap v478: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:57:17.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:17.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[29507]: osdmap e345: 8 total, 8 up, 8 in 2024-02-05T07:57:17.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:17.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-82060-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-82060-64"}]': finished 2024-02-05T07:57:17.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-49"}]': finished 2024-02-05T07:57:17.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[29507]: osdmap e346: 8 total, 8 up, 8 in 2024-02-05T07:57:17.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-49", "mode": "readproxy"}]: dispatch 2024-02-05T07:57:17.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-49", "mode": "readproxy"}]: dispatch 2024-02-05T07:57:17.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[41654]: pgmap v478: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:57:17.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:17.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[41654]: osdmap e345: 8 total, 8 up, 8 in 2024-02-05T07:57:17.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:17.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-82060-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-82060-64"}]': finished 2024-02-05T07:57:17.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-49"}]': finished 2024-02-05T07:57:17.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[41654]: osdmap e346: 8 total, 8 up, 8 in 2024-02-05T07:57:17.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-49", "mode": "readproxy"}]: dispatch 2024-02-05T07:57:17.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-49", "mode": "readproxy"}]: dispatch 2024-02-05T07:57:18.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:18 smithi157 ceph-mon[33345]: pgmap v481: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:57:18.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:18 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:57:18.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-49", "mode": "readproxy"}]': finished 2024-02-05T07:57:18.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:18 smithi157 ceph-mon[33345]: osdmap e347: 8 total, 8 up, 8 in 2024-02-05T07:57:18.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:18 smithi131 ceph-mon[29507]: pgmap v481: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:57:18.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:18 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:57:18.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-49", "mode": "readproxy"}]': finished 2024-02-05T07:57:18.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:18 smithi131 ceph-mon[29507]: osdmap e347: 8 total, 8 up, 8 in 2024-02-05T07:57:18.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:18 smithi131 ceph-mon[41654]: pgmap v481: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:57:18.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:18 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:57:18.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-49", "mode": "readproxy"}]': finished 2024-02-05T07:57:18.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:18 smithi131 ceph-mon[41654]: osdmap e347: 8 total, 8 up, 8 in 2024-02-05T07:57:18.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:57:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:57:19.419 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:19 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:19.419 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:19 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:19.420 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:19 smithi131 ceph-mon[41654]: osdmap e348: 8 total, 8 up, 8 in 2024-02-05T07:57:19.420 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:19.420 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:19.420 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:19 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:19.420 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:19 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:19.420 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:19 smithi131 ceph-mon[29507]: osdmap e348: 8 total, 8 up, 8 in 2024-02-05T07:57:19.420 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:19.420 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:19.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:19 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:19.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:19 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:19.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:19 smithi157 ceph-mon[33345]: osdmap e348: 8 total, 8 up, 8 in 2024-02-05T07:57:19.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:19.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:20 smithi157 ceph-mon[33345]: pgmap v484: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:57:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:20 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-82060-64"}]': finished 2024-02-05T07:57:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:20 smithi157 ceph-mon[33345]: osdmap e349: 8 total, 8 up, 8 in 2024-02-05T07:57:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:20 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:21.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:20 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:20 smithi131 ceph-mon[41654]: pgmap v484: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:57:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:20 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-82060-64"}]': finished 2024-02-05T07:57:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:20 smithi131 ceph-mon[41654]: osdmap e349: 8 total, 8 up, 8 in 2024-02-05T07:57:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:20 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:20 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:21.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:20 smithi131 ceph-mon[29507]: pgmap v484: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:57:21.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:20 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-82060-64"}]': finished 2024-02-05T07:57:21.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:20 smithi131 ceph-mon[29507]: osdmap e349: 8 total, 8 up, 8 in 2024-02-05T07:57:21.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:20 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/872255397' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:21.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:20 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-82060-64"}]: dispatch 2024-02-05T07:57:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-82060-64"}]': finished 2024-02-05T07:57:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:21 smithi157 ceph-mon[33345]: osdmap e350: 8 total, 8 up, 8 in 2024-02-05T07:57:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:22.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:22.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:22.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-82060-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:22.224 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-82060-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:22.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-82060-64"}]': finished 2024-02-05T07:57:22.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:21 smithi131 ceph-mon[41654]: osdmap e350: 8 total, 8 up, 8 in 2024-02-05T07:57:22.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:22.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:22.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:22.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:22.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-82060-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:22.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-82060-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-82060-64"}]': finished 2024-02-05T07:57:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:21 smithi131 ceph-mon[29507]: osdmap e350: 8 total, 8 up, 8 in 2024-02-05T07:57:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:22.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-82060-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:22.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-82060-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:23.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:57:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:57:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:57:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:23 smithi131 ceph-mon[41654]: pgmap v487: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:57:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-82060-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:57:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:23 smithi131 ceph-mon[41654]: osdmap e351: 8 total, 8 up, 8 in 2024-02-05T07:57:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-82060-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:23.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-82060-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:23 smithi131 ceph-mon[29507]: pgmap v487: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:57:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-82060-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:57:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:23 smithi131 ceph-mon[29507]: osdmap e351: 8 total, 8 up, 8 in 2024-02-05T07:57:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-82060-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-82060-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:23.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:23 smithi157 ceph-mon[33345]: pgmap v487: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:57:23.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-82060-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:57:23.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:23 smithi157 ceph-mon[33345]: osdmap e351: 8 total, 8 up, 8 in 2024-02-05T07:57:23.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-82060-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:23.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-82060-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:24.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:24 smithi157 ceph-mon[33345]: osdmap e352: 8 total, 8 up, 8 in 2024-02-05T07:57:24.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:24 smithi157 ceph-mon[33345]: pgmap v490: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:24.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-82060-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-82060-65"}]': finished 2024-02-05T07:57:24.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:24 smithi157 ceph-mon[33345]: osdmap e353: 8 total, 8 up, 8 in 2024-02-05T07:57:24.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:24 smithi131 ceph-mon[41654]: osdmap e352: 8 total, 8 up, 8 in 2024-02-05T07:57:24.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:24 smithi131 ceph-mon[41654]: pgmap v490: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:24.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-82060-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-82060-65"}]': finished 2024-02-05T07:57:24.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:24 smithi131 ceph-mon[41654]: osdmap e353: 8 total, 8 up, 8 in 2024-02-05T07:57:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:24 smithi131 ceph-mon[29507]: osdmap e352: 8 total, 8 up, 8 in 2024-02-05T07:57:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:24 smithi131 ceph-mon[29507]: pgmap v490: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-82060-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-82060-65"}]': finished 2024-02-05T07:57:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:24 smithi131 ceph-mon[29507]: osdmap e353: 8 total, 8 up, 8 in 2024-02-05T07:57:26.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:25 smithi157 ceph-mon[33345]: osdmap e354: 8 total, 8 up, 8 in 2024-02-05T07:57:26.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:25 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:26.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:25 smithi131 ceph-mon[41654]: osdmap e354: 8 total, 8 up, 8 in 2024-02-05T07:57:26.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:25 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:26.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:25 smithi131 ceph-mon[29507]: osdmap e354: 8 total, 8 up, 8 in 2024-02-05T07:57:26.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:25 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:26 smithi157 ceph-mon[33345]: pgmap v493: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:26 smithi157 ceph-mon[33345]: osdmap e355: 8 total, 8 up, 8 in 2024-02-05T07:57:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:26 smithi131 ceph-mon[41654]: pgmap v493: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:26 smithi131 ceph-mon[41654]: osdmap e355: 8 total, 8 up, 8 in 2024-02-05T07:57:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:26 smithi131 ceph-mon[29507]: pgmap v493: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:26 smithi131 ceph-mon[29507]: osdmap e355: 8 total, 8 up, 8 in 2024-02-05T07:57:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:28.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-82060-65"}]': finished 2024-02-05T07:57:28.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:27 smithi157 ceph-mon[33345]: osdmap e356: 8 total, 8 up, 8 in 2024-02-05T07:57:28.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:27 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:28.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-82060-65"}]': finished 2024-02-05T07:57:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:27 smithi131 ceph-mon[41654]: osdmap e356: 8 total, 8 up, 8 in 2024-02-05T07:57:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:27 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:28.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-82060-65"}]': finished 2024-02-05T07:57:28.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:27 smithi131 ceph-mon[29507]: osdmap e356: 8 total, 8 up, 8 in 2024-02-05T07:57:28.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:27 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/327621174' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:28.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-82060-65"}]: dispatch 2024-02-05T07:57:28.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:57:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:57:28.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:28 smithi157 ceph-mon[33345]: pgmap v496: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:57:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-82060-65"}]': finished 2024-02-05T07:57:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:28 smithi157 ceph-mon[33345]: osdmap e357: 8 total, 8 up, 8 in 2024-02-05T07:57:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi131-82060-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi131-82060-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:28.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[41654]: pgmap v496: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:57:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-82060-65"}]': finished 2024-02-05T07:57:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[41654]: osdmap e357: 8 total, 8 up, 8 in 2024-02-05T07:57:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:29.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:29.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:29.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi131-82060-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:29.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi131-82060-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:29.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:29.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:29.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[29507]: pgmap v496: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:57:29.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-82060-65"}]': finished 2024-02-05T07:57:29.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[29507]: osdmap e357: 8 total, 8 up, 8 in 2024-02-05T07:57:29.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:29.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:29.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:29.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:29.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi131-82060-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:29.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi131-82060-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:57:29.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:29.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:29 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi131-82060-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:57:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:57:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:29 smithi157 ceph-mon[33345]: osdmap e358: 8 total, 8 up, 8 in 2024-02-05T07:57:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi131-82060-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi131-82060-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:29 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:57:30.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:29 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi131-82060-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:57:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:57:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[41654]: osdmap e358: 8 total, 8 up, 8 in 2024-02-05T07:57:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi131-82060-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi131-82060-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:57:30.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi131-82060-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:57:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:57:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[29507]: osdmap e358: 8 total, 8 up, 8 in 2024-02-05T07:57:30.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi131-82060-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:30.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:30.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi131-82060-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:30.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:57:30.270 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:29 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:30.895 INFO:tasks.workunit.client.0.smithi131.stdout:163:head 2024-02-05T07:57:30.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:5d165639:::164:head 2024-02-05T07:57:30.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:f43765fc:::165:head 2024-02-05T07:57:30.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:b4c720e9:::166:head 2024-02-05T07:57:30.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:e694b040:::167:head 2024-02-05T07:57:30.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:afa38db2:::168:head 2024-02-05T07:57:30.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:77ba9f53:::169:head 2024-02-05T07:57:30.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:87495034:::170:head 2024-02-05T07:57:30.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:7c96bf0e:::171:head 2024-02-05T07:57:30.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:dbe346cc:::172:head 2024-02-05T07:57:30.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:e943ec24:::173:head 2024-02-05T07:57:30.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:f97a9c0c:::174:head 2024-02-05T07:57:30.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:6f26e74d:::175:head 2024-02-05T07:57:30.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:4f95e106:::176:head 2024-02-05T07:57:30.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:0e6f2f8f:::177:head 2024-02-05T07:57:30.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:05db05f1:::178:head 2024-02-05T07:57:30.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:38a78d66:::179:head 2024-02-05T07:57:30.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:d095610b:::180:head 2024-02-05T07:57:30.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:a1a9d709:::181:head 2024-02-05T07:57:30.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:1e5d39db:::182:head 2024-02-05T07:57:30.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:f7df4fb9:::183:head 2024-02-05T07:57:30.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:03a7f161:::184:head 2024-02-05T07:57:30.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:ba70721e:::185:head 2024-02-05T07:57:30.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:28e5662d:::186:head 2024-02-05T07:57:30.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:973d52de:::187:head 2024-02-05T07:57:30.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:4303eb1c:::188:head 2024-02-05T07:57:30.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:b990b48e:::189:head 2024-02-05T07:57:30.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:29b8165b:::190:head 2024-02-05T07:57:30.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:3547f197:::191:head 2024-02-05T07:57:30.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:7e260936:::192:head 2024-02-05T07:57:30.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:1abec7b1:::193:head 2024-02-05T07:57:30.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:10fdda93:::194:head 2024-02-05T07:57:30.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:15817eea:::195:head 2024-02-05T07:57:30.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:770bab57:::196:head 2024-02-05T07:57:30.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:ed9e13e7:::197:head 2024-02-05T07:57:30.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:71471a8f:::198:head 2024-02-05T07:57:30.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 276:10fb1d02:::199:head 2024-02-05T07:57:30.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8282 ms) 2024-02-05T07:57:30.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-02-05T07:57:30.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707119805,0 2024-02-05T07:57:30.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: first is 1707119805 2024-02-05T07:57:30.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707119805,0 2024-02-05T07:57:30.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707119805,0 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707119805,1707119807,1707119808,0 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707119805,1707119807,1707119808,0 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707119805,1707119807,1707119808,0 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707119805,1707119807,1707119808,1707119810,1707119811,0 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707119805,1707119807,1707119808,1707119810,1707119811,0 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707119805,1707119807,1707119808,1707119810,1707119811,0 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707119805,1707119807,1707119808,1707119810,1707119811,1707119813,1707119814,0 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707119805,1707119807,1707119808,1707119810,1707119811,1707119813,1707119814,0 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707119805,1707119807,1707119808,1707119810,1707119811,1707119813,1707119814,0 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707119808,1707119810,1707119811,1707119813,1707119814,1707119816,1707119817,0 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: first now 1707119808, trimmed 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (20856 ms) 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: foo0 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14036 ms) 2024-02-05T07:57:30.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-02-05T07:57:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:30 smithi157 ceph-mon[33345]: pgmap v499: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:30 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:57:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49"}]': finished 2024-02-05T07:57:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:30 smithi157 ceph-mon[33345]: osdmap e359: 8 total, 8 up, 8 in 2024-02-05T07:57:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:31.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:30 smithi131 ceph-mon[41654]: pgmap v499: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:30 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:57:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49"}]': finished 2024-02-05T07:57:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:30 smithi131 ceph-mon[41654]: osdmap e359: 8 total, 8 up, 8 in 2024-02-05T07:57:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:31.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:31.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:30 smithi131 ceph-mon[29507]: pgmap v499: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:31.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:30 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:57:31.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49"}]': finished 2024-02-05T07:57:31.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:30 smithi131 ceph-mon[29507]: osdmap e359: 8 total, 8 up, 8 in 2024-02-05T07:57:31.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:31.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:31.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:31.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-49"}]: dispatch 2024-02-05T07:57:32.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:31 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi131-82060-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi131-82060-66"}]': finished 2024-02-05T07:57:32.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:31 smithi157 ceph-mon[33345]: osdmap e360: 8 total, 8 up, 8 in 2024-02-05T07:57:32.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:31 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:32.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:31 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi131-82060-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi131-82060-66"}]': finished 2024-02-05T07:57:32.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:31 smithi131 ceph-mon[41654]: osdmap e360: 8 total, 8 up, 8 in 2024-02-05T07:57:32.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:31 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:32.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:31 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi131-82060-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi131-82060-66"}]': finished 2024-02-05T07:57:32.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:31 smithi131 ceph-mon[29507]: osdmap e360: 8 total, 8 up, 8 in 2024-02-05T07:57:32.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:31 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:32 smithi157 ceph-mon[33345]: pgmap v502: 268 pgs: 8 unknown, 260 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:32 smithi157 ceph-mon[33345]: osdmap e361: 8 total, 8 up, 8 in 2024-02-05T07:57:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:32 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:33.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:33.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:57:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:57:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:57:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:32 smithi131 ceph-mon[29507]: pgmap v502: 268 pgs: 8 unknown, 260 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:32 smithi131 ceph-mon[29507]: osdmap e361: 8 total, 8 up, 8 in 2024-02-05T07:57:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:32 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:32 smithi131 ceph-mon[41654]: pgmap v502: 268 pgs: 8 unknown, 260 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:32 smithi131 ceph-mon[41654]: osdmap e361: 8 total, 8 up, 8 in 2024-02-05T07:57:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:32 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:33.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:34.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:57:34.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:33 smithi157 ceph-mon[33345]: osdmap e362: 8 total, 8 up, 8 in 2024-02-05T07:57:34.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:33 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:34.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:34.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:33 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:34.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:34.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:57:34.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:33 smithi131 ceph-mon[29507]: osdmap e362: 8 total, 8 up, 8 in 2024-02-05T07:57:34.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:33 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:34.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:34.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:33 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:34.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:34.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:57:34.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:33 smithi131 ceph-mon[41654]: osdmap e362: 8 total, 8 up, 8 in 2024-02-05T07:57:34.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:33 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:34.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:34.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:33 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:34.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:57:34.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3027 ms) 2024-02-05T07:57:34.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-02-05T07:57:34.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7034 ms) 2024-02-05T07:57:34.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-02-05T07:57:34.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7010 ms) 2024-02-05T07:57:34.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-02-05T07:57:34.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7030 ms) 2024-02-05T07:57:34.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-02-05T07:57:34.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7022 ms) 2024-02-05T07:57:34.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-02-05T07:57:34.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7038 ms) 2024-02-05T07:57:34.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-02-05T07:57:34.932 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7049 ms) 2024-02-05T07:57:34.932 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139128 ms total) 2024-02-05T07:57:34.932 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: 2024-02-05T07:57:34.932 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-02-05T07:57:34.932 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (296219 ms total) 2024-02-05T07:57:34.932 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-02-05T07:57:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:34 smithi157 ceph-mon[33345]: pgmap v505: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:57:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-82060-66"}]': finished 2024-02-05T07:57:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:57:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:34 smithi157 ceph-mon[33345]: osdmap e363: 8 total, 8 up, 8 in 2024-02-05T07:57:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:35.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:35.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:34 smithi131 ceph-mon[29507]: pgmap v505: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:57:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-82060-66"}]': finished 2024-02-05T07:57:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:57:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:35.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:34 smithi131 ceph-mon[29507]: osdmap e363: 8 total, 8 up, 8 in 2024-02-05T07:57:35.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:35.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:35.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:35.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:34 smithi131 ceph-mon[41654]: pgmap v505: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:57:35.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-82060-66"}]': finished 2024-02-05T07:57:35.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:57:35.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:35.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:34 smithi131 ceph-mon[41654]: osdmap e363: 8 total, 8 up, 8 in 2024-02-05T07:57:35.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:35.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2940202746' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:35.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-82060-66"}]: dispatch 2024-02-05T07:57:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-51"}]': finished 2024-02-05T07:57:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-82060-66"}]': finished 2024-02-05T07:57:36.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:35 smithi157 ceph-mon[33345]: osdmap e364: 8 total, 8 up, 8 in 2024-02-05T07:57:36.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-51", "mode": "writeback"}]: dispatch 2024-02-05T07:57:36.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-51", "mode": "writeback"}]: dispatch 2024-02-05T07:57:36.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:35 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:57:36.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-51"}]': finished 2024-02-05T07:57:36.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-82060-66"}]': finished 2024-02-05T07:57:36.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:35 smithi131 ceph-mon[41654]: osdmap e364: 8 total, 8 up, 8 in 2024-02-05T07:57:36.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-51", "mode": "writeback"}]: dispatch 2024-02-05T07:57:36.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-51", "mode": "writeback"}]: dispatch 2024-02-05T07:57:36.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:35 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:57:36.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-51"}]': finished 2024-02-05T07:57:36.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-82060-66"}]': finished 2024-02-05T07:57:36.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:35 smithi131 ceph-mon[29507]: osdmap e364: 8 total, 8 up, 8 in 2024-02-05T07:57:36.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-51", "mode": "writeback"}]: dispatch 2024-02-05T07:57:36.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-51", "mode": "writeback"}]: dispatch 2024-02-05T07:57:36.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:35 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:57:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:36 smithi157 ceph-mon[33345]: pgmap v508: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:57:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-51", "mode": "writeback"}]': finished 2024-02-05T07:57:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:36 smithi157 ceph-mon[33345]: osdmap e365: 8 total, 8 up, 8 in 2024-02-05T07:57:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:57:37.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:57:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:36 smithi131 ceph-mon[41654]: pgmap v508: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:57:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-51", "mode": "writeback"}]': finished 2024-02-05T07:57:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:36 smithi131 ceph-mon[41654]: osdmap e365: 8 total, 8 up, 8 in 2024-02-05T07:57:37.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:57:37.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:57:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:36 smithi131 ceph-mon[29507]: pgmap v508: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:57:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-51", "mode": "writeback"}]': finished 2024-02-05T07:57:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:36 smithi131 ceph-mon[29507]: osdmap e365: 8 total, 8 up, 8 in 2024-02-05T07:57:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:57:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:57:38.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:57:38.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:37 smithi157 ceph-mon[33345]: osdmap e366: 8 total, 8 up, 8 in 2024-02-05T07:57:38.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:57:38.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:57:38.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:57:38.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:37 smithi131 ceph-mon[41654]: osdmap e366: 8 total, 8 up, 8 in 2024-02-05T07:57:38.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:57:38.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:57:38.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:57:38.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:37 smithi131 ceph-mon[29507]: osdmap e366: 8 total, 8 up, 8 in 2024-02-05T07:57:38.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:57:38.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:57:38.954 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:57:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:57:39.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:38 smithi157 ceph-mon[33345]: pgmap v511: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:57:39.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:57:39.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:38 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:57:39.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:38 smithi157 ceph-mon[33345]: osdmap e367: 8 total, 8 up, 8 in 2024-02-05T07:57:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:57:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:38 smithi131 ceph-mon[41654]: pgmap v511: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:57:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:57:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:38 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:57:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:38 smithi131 ceph-mon[41654]: osdmap e367: 8 total, 8 up, 8 in 2024-02-05T07:57:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:57:39.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:38 smithi131 ceph-mon[29507]: pgmap v511: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:57:39.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:57:39.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:38 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:57:39.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:38 smithi131 ceph-mon[29507]: osdmap e367: 8 total, 8 up, 8 in 2024-02-05T07:57:39.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:57:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:39 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:39 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:57:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:57:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:39 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:57:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:39 smithi157 ceph-mon[33345]: osdmap e368: 8 total, 8 up, 8 in 2024-02-05T07:57:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:57:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:57:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:39 smithi157 ceph-mon[33345]: osdmap e369: 8 total, 8 up, 8 in 2024-02-05T07:57:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:39 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:57:40.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:57:40.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:40.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:57:40.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:57:40.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:57:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[29507]: osdmap e368: 8 total, 8 up, 8 in 2024-02-05T07:57:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:57:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:57:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[29507]: osdmap e369: 8 total, 8 up, 8 in 2024-02-05T07:57:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:57:40.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:57:40.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:40.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:57:40.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:57:40.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:57:40.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[41654]: osdmap e368: 8 total, 8 up, 8 in 2024-02-05T07:57:40.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:57:40.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:57:40.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[41654]: osdmap e369: 8 total, 8 up, 8 in 2024-02-05T07:57:40.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:57:40.270 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:57:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:40 smithi157 ceph-mon[33345]: pgmap v514: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:57:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "target_max_objects","val": "1"}]': finished 2024-02-05T07:57:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:40 smithi157 ceph-mon[33345]: osdmap e370: 8 total, 8 up, 8 in 2024-02-05T07:57:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:40 smithi131 ceph-mon[41654]: pgmap v514: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:57:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "target_max_objects","val": "1"}]': finished 2024-02-05T07:57:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:40 smithi131 ceph-mon[41654]: osdmap e370: 8 total, 8 up, 8 in 2024-02-05T07:57:41.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:40 smithi131 ceph-mon[29507]: pgmap v514: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:57:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-51","var": "target_max_objects","val": "1"}]': finished 2024-02-05T07:57:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:40 smithi131 ceph-mon[29507]: osdmap e370: 8 total, 8 up, 8 in 2024-02-05T07:57:42.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:42 smithi131 ceph-mon[41654]: pgmap v517: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:57:42.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:42 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-05T07:57:42.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:42 smithi131 ceph-mon[29507]: pgmap v517: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:57:42.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:42 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-05T07:57:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:42 smithi157 ceph-mon[33345]: pgmap v517: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:57:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:42 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-05T07:57:43.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:57:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:57:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:57:44.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:44 smithi157 ceph-mon[33345]: pgmap v518: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.6 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-05T07:57:44.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:44 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:57:44.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:44 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:44 smithi131 ceph-mon[41654]: pgmap v518: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.6 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-05T07:57:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:44 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:57:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:44 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:45.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:44 smithi131 ceph-mon[29507]: pgmap v518: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.6 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-05T07:57:45.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:44 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:57:45.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:44 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:46.683 INFO:tasks.workunit.client.0.smithi131.stdout: started 0x559251034170 2024-02-05T07:57:46.683 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251034ae0 2024-02-05T07:57:46.683 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510344f0 2024-02-05T07:57:46.683 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251034640 2024-02-05T07:57:46.683 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251034750 2024-02-05T07:57:46.683 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251034bf0 2024-02-05T07:57:46.683 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251035060 2024-02-05T07:57:46.684 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251035170 2024-02-05T07:57:46.684 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251035760 2024-02-05T07:57:46.684 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251034d00 2024-02-05T07:57:46.684 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251034e30 2024-02-05T07:57:46.684 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510352c0 2024-02-05T07:57:46.684 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510353f0 2024-02-05T07:57:46.684 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251035500 2024-02-05T07:57:46.684 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251035610 2024-02-05T07:57:46.684 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251035d50 2024-02-05T07:57:46.684 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510358b0 2024-02-05T07:57:46.684 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510359e0 2024-02-05T07:57:46.684 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251035af0 2024-02-05T07:57:46.684 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251035c40 2024-02-05T07:57:46.684 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251036320 2024-02-05T07:57:46.684 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251035e80 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251035fb0 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510360c0 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251036210 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251036910 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251036470 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510365a0 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510366b0 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251036800 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251036f00 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251036a60 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251036b90 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251036ca0 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251036db0 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510374f0 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251037050 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251037160 2024-02-05T07:57:46.685 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251037290 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510373e0 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251037ae0 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251037640 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251037770 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510378a0 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510379d0 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251038090 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251037bf0 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251037d40 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251037e50 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251037f80 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251038660 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510381c0 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510382f0 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251038400 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251038550 2024-02-05T07:57:46.686 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251038c30 2024-02-05T07:57:46.687 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251038790 2024-02-05T07:57:46.687 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510388c0 2024-02-05T07:57:46.687 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251038a10 2024-02-05T07:57:46.687 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251038d80 2024-02-05T07:57:46.687 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251038ed0 2024-02-05T07:57:46.687 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251038b20 2024-02-05T07:57:46.687 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251039380 2024-02-05T07:57:46.687 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251039020 2024-02-05T07:57:46.687 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251039150 2024-02-05T07:57:46.687 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251039490 2024-02-05T07:57:46.687 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251039920 2024-02-05T07:57:46.687 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251039a70 2024-02-05T07:57:46.687 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102f6b0 2024-02-05T07:57:46.687 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102f7c0 2024-02-05T07:57:46.687 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925102f8f0 2024-02-05T07:57:46.688 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251039bc0 2024-02-05T07:57:46.688 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103a050 2024-02-05T07:57:46.688 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103a180 2024-02-05T07:57:46.688 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103a730 2024-02-05T07:57:46.688 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251039cf0 2024-02-05T07:57:46.688 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251039e40 2024-02-05T07:57:46.688 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103a290 2024-02-05T07:57:46.688 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103a3c0 2024-02-05T07:57:46.688 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103a4f0 2024-02-05T07:57:46.688 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103a880 2024-02-05T07:57:46.688 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103ad60 2024-02-05T07:57:46.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:46 smithi157 ceph-mon[33345]: pgmap v519: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 791 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:57:47.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:46 smithi131 ceph-mon[41654]: pgmap v519: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 791 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:57:47.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:46 smithi131 ceph-mon[29507]: pgmap v519: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 791 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:57:48.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:48 smithi157 ceph-mon[33345]: pgmap v520: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-02-05T07:57:48.972 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:57:48 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:57:49.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:48 smithi131 ceph-mon[41654]: pgmap v520: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-02-05T07:57:49.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:48 smithi131 ceph-mon[29507]: pgmap v520: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-02-05T07:57:49.919 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:49 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:49.919 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:49 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:49 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:50 smithi157 ceph-mon[33345]: pgmap v521: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:57:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:50 smithi131 ceph-mon[41654]: pgmap v521: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:57:51.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:50 smithi131 ceph-mon[29507]: pgmap v521: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:57:51.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:51.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:52.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:52.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:52 smithi131 ceph-mon[41654]: pgmap v522: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 978 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:57:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:57:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:52 smithi131 ceph-mon[41654]: osdmap e371: 8 total, 8 up, 8 in 2024-02-05T07:57:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:52.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:52 smithi131 ceph-mon[29507]: pgmap v522: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 978 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:57:52.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:57:52.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:52.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:52 smithi131 ceph-mon[29507]: osdmap e371: 8 total, 8 up, 8 in 2024-02-05T07:57:52.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:52 smithi157 ceph-mon[33345]: pgmap v522: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 978 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:57:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T07:57:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:52 smithi157 ceph-mon[33345]: osdmap e371: 8 total, 8 up, 8 in 2024-02-05T07:57:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:53.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:57:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:57:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:57:53.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51"}]': finished 2024-02-05T07:57:53.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:53 smithi157 ceph-mon[33345]: osdmap e372: 8 total, 8 up, 8 in 2024-02-05T07:57:53.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:53.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:53.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:53.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:54.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51"}]': finished 2024-02-05T07:57:54.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:53 smithi131 ceph-mon[29507]: osdmap e372: 8 total, 8 up, 8 in 2024-02-05T07:57:54.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:54.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:54.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:54.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51"}]': finished 2024-02-05T07:57:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:53 smithi131 ceph-mon[41654]: osdmap e372: 8 total, 8 up, 8 in 2024-02-05T07:57:54.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:54.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:54.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:54.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-51"}]: dispatch 2024-02-05T07:57:54.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:54 smithi157 ceph-mon[33345]: pgmap v525: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:57:54.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:54 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-05T07:57:54.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:54 smithi157 ceph-mon[33345]: osdmap e373: 8 total, 8 up, 8 in 2024-02-05T07:57:55.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:54 smithi131 ceph-mon[41654]: pgmap v525: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:57:55.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:54 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-05T07:57:55.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:54 smithi131 ceph-mon[41654]: osdmap e373: 8 total, 8 up, 8 in 2024-02-05T07:57:55.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:54 smithi131 ceph-mon[29507]: pgmap v525: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:57:55.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:54 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-05T07:57:55.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:54 smithi131 ceph-mon[29507]: osdmap e373: 8 total, 8 up, 8 in 2024-02-05T07:57:55.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:55 smithi157 ceph-mon[33345]: osdmap e374: 8 total, 8 up, 8 in 2024-02-05T07:57:55.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:55.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:56.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:55 smithi131 ceph-mon[41654]: osdmap e374: 8 total, 8 up, 8 in 2024-02-05T07:57:56.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:56.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:56.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:55 smithi131 ceph-mon[29507]: osdmap e374: 8 total, 8 up, 8 in 2024-02-05T07:57:56.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:56.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:56 smithi157 ceph-mon[33345]: pgmap v528: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-05T07:57:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:56 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:57:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:56 smithi157 ceph-mon[33345]: osdmap e375: 8 total, 8 up, 8 in 2024-02-05T07:57:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-53"}]: dispatch 2024-02-05T07:57:57.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-53"}]: dispatch 2024-02-05T07:57:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:56 smithi131 ceph-mon[41654]: pgmap v528: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-05T07:57:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:56 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:57:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:56 smithi131 ceph-mon[41654]: osdmap e375: 8 total, 8 up, 8 in 2024-02-05T07:57:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-53"}]: dispatch 2024-02-05T07:57:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-53"}]: dispatch 2024-02-05T07:57:57.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:56 smithi131 ceph-mon[29507]: pgmap v528: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-05T07:57:57.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:56 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:57:57.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:57:57.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:56 smithi131 ceph-mon[29507]: osdmap e375: 8 total, 8 up, 8 in 2024-02-05T07:57:57.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:57.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:57.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-53"}]: dispatch 2024-02-05T07:57:57.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-53"}]: dispatch 2024-02-05T07:57:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:57 smithi157 ceph-mon[33345]: osdmap e376: 8 total, 8 up, 8 in 2024-02-05T07:57:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:57 smithi157 ceph-mon[33345]: osdmap e377: 8 total, 8 up, 8 in 2024-02-05T07:57:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:57 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:57 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:58.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:57 smithi131 ceph-mon[41654]: osdmap e376: 8 total, 8 up, 8 in 2024-02-05T07:57:58.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:57 smithi131 ceph-mon[41654]: osdmap e377: 8 total, 8 up, 8 in 2024-02-05T07:57:58.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:57 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:58.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:57 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:58.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:57 smithi131 ceph-mon[29507]: osdmap e376: 8 total, 8 up, 8 in 2024-02-05T07:57:58.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:57 smithi131 ceph-mon[29507]: osdmap e377: 8 total, 8 up, 8 in 2024-02-05T07:57:58.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:57 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:58.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:57 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:57:58.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:57:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:57:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:58 smithi157 ceph-mon[33345]: pgmap v531: 260 pgs: 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:57:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:57:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:58 smithi157 ceph-mon[33345]: osdmap e378: 8 total, 8 up, 8 in 2024-02-05T07:57:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:57:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-55"}]: dispatch 2024-02-05T07:57:58.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-55"}]: dispatch 2024-02-05T07:57:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:58 smithi131 ceph-mon[41654]: pgmap v531: 260 pgs: 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:57:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:57:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:58 smithi131 ceph-mon[41654]: osdmap e378: 8 total, 8 up, 8 in 2024-02-05T07:57:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:57:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-55"}]: dispatch 2024-02-05T07:57:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-55"}]: dispatch 2024-02-05T07:57:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:58 smithi131 ceph-mon[29507]: pgmap v531: 260 pgs: 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:57:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:57:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:58 smithi131 ceph-mon[29507]: osdmap e378: 8 total, 8 up, 8 in 2024-02-05T07:57:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:57:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:57:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-55"}]: dispatch 2024-02-05T07:57:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-55"}]: dispatch 2024-02-05T07:58:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:59 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:59 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:58:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:59 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:00.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:57:59 smithi157 ceph-mon[33345]: osdmap e379: 8 total, 8 up, 8 in 2024-02-05T07:58:00.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:59 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:00.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:59 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:58:00.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:59 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:00.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:57:59 smithi131 ceph-mon[41654]: osdmap e379: 8 total, 8 up, 8 in 2024-02-05T07:58:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:59 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:59 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:58:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:59 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:57:59 smithi131 ceph-mon[29507]: osdmap e379: 8 total, 8 up, 8 in 2024-02-05T07:58:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:00 smithi157 ceph-mon[33345]: pgmap v534: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:58:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:00 smithi157 ceph-mon[33345]: osdmap e380: 8 total, 8 up, 8 in 2024-02-05T07:58:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:00 smithi131 ceph-mon[41654]: pgmap v534: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:58:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:00 smithi131 ceph-mon[41654]: osdmap e380: 8 total, 8 up, 8 in 2024-02-05T07:58:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:00 smithi131 ceph-mon[29507]: pgmap v534: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:58:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:00 smithi131 ceph-mon[29507]: osdmap e380: 8 total, 8 up, 8 in 2024-02-05T07:58:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:01 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:58:02.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:58:02.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:01 smithi157 ceph-mon[33345]: osdmap e381: 8 total, 8 up, 8 in 2024-02-05T07:58:02.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:02.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:02.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-57"}]: dispatch 2024-02-05T07:58:02.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-57"}]: dispatch 2024-02-05T07:58:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:01 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:58:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:58:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:01 smithi131 ceph-mon[41654]: osdmap e381: 8 total, 8 up, 8 in 2024-02-05T07:58:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-57"}]: dispatch 2024-02-05T07:58:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-57"}]: dispatch 2024-02-05T07:58:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:01 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:58:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:58:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:01 smithi131 ceph-mon[29507]: osdmap e381: 8 total, 8 up, 8 in 2024-02-05T07:58:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-57"}]: dispatch 2024-02-05T07:58:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-57"}]: dispatch 2024-02-05T07:58:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:02 smithi157 ceph-mon[33345]: pgmap v537: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:58:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:02 smithi157 ceph-mon[33345]: osdmap e382: 8 total, 8 up, 8 in 2024-02-05T07:58:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:02 smithi131 ceph-mon[41654]: pgmap v537: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:58:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:02 smithi131 ceph-mon[41654]: osdmap e382: 8 total, 8 up, 8 in 2024-02-05T07:58:03.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:58:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:58:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:58:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:02 smithi131 ceph-mon[29507]: pgmap v537: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:58:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:02 smithi131 ceph-mon[29507]: osdmap e382: 8 total, 8 up, 8 in 2024-02-05T07:58:04.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:04 smithi157 ceph-mon[33345]: pgmap v540: 260 pgs: 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:58:04.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:04 smithi157 ceph-mon[33345]: osdmap e383: 8 total, 8 up, 8 in 2024-02-05T07:58:04.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:04.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:05.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:04 smithi131 ceph-mon[29507]: pgmap v540: 260 pgs: 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:58:05.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:04 smithi131 ceph-mon[29507]: osdmap e383: 8 total, 8 up, 8 in 2024-02-05T07:58:05.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:05.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:05.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:04 smithi131 ceph-mon[41654]: pgmap v540: 260 pgs: 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:58:05.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:04 smithi131 ceph-mon[41654]: osdmap e383: 8 total, 8 up, 8 in 2024-02-05T07:58:05.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:05.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:05.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:58:05.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:05 smithi157 ceph-mon[33345]: osdmap e384: 8 total, 8 up, 8 in 2024-02-05T07:58:05.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:58:05.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:05.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:05.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-59"}]: dispatch 2024-02-05T07:58:05.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-59"}]: dispatch 2024-02-05T07:58:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:58:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:05 smithi131 ceph-mon[41654]: osdmap e384: 8 total, 8 up, 8 in 2024-02-05T07:58:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:58:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-59"}]: dispatch 2024-02-05T07:58:06.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-59"}]: dispatch 2024-02-05T07:58:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:58:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:05 smithi131 ceph-mon[29507]: osdmap e384: 8 total, 8 up, 8 in 2024-02-05T07:58:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:58:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-59"}]: dispatch 2024-02-05T07:58:06.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-59"}]: dispatch 2024-02-05T07:58:06.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:06 smithi157 ceph-mon[33345]: pgmap v543: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:58:06.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:06 smithi157 ceph-mon[33345]: osdmap e385: 8 total, 8 up, 8 in 2024-02-05T07:58:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:06 smithi131 ceph-mon[41654]: pgmap v543: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:58:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:06 smithi131 ceph-mon[41654]: osdmap e385: 8 total, 8 up, 8 in 2024-02-05T07:58:07.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:06 smithi131 ceph-mon[29507]: pgmap v543: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:58:07.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:06 smithi131 ceph-mon[29507]: osdmap e385: 8 total, 8 up, 8 in 2024-02-05T07:58:07.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:07 smithi157 ceph-mon[33345]: osdmap e386: 8 total, 8 up, 8 in 2024-02-05T07:58:07.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:07 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:07.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:08.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:07 smithi131 ceph-mon[41654]: osdmap e386: 8 total, 8 up, 8 in 2024-02-05T07:58:08.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:07 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:08.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:08.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:07 smithi131 ceph-mon[29507]: osdmap e386: 8 total, 8 up, 8 in 2024-02-05T07:58:08.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:07 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:08.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:08.972 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:58:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:58:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:08 smithi157 ceph-mon[33345]: pgmap v546: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:58:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:08 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:58:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:58:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:08 smithi157 ceph-mon[33345]: osdmap e387: 8 total, 8 up, 8 in 2024-02-05T07:58:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:58:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:08.973 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:08 smithi131 ceph-mon[41654]: pgmap v546: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:58:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:08 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:58:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:58:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:08 smithi131 ceph-mon[41654]: osdmap e387: 8 total, 8 up, 8 in 2024-02-05T07:58:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:58:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:09.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:08 smithi131 ceph-mon[29507]: pgmap v546: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:58:09.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:08 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:58:09.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:58:09.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:08 smithi131 ceph-mon[29507]: osdmap e387: 8 total, 8 up, 8 in 2024-02-05T07:58:09.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:58:09.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:09.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:09.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:58:09.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:09 smithi157 ceph-mon[33345]: osdmap e388: 8 total, 8 up, 8 in 2024-02-05T07:58:09.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:09 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:09.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:09.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:09.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-61"}]: dispatch 2024-02-05T07:58:09.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-61"}]: dispatch 2024-02-05T07:58:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:58:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:09 smithi131 ceph-mon[29507]: osdmap e388: 8 total, 8 up, 8 in 2024-02-05T07:58:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:09 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-61"}]: dispatch 2024-02-05T07:58:10.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-61"}]: dispatch 2024-02-05T07:58:10.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:58:10.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:09 smithi131 ceph-mon[41654]: osdmap e388: 8 total, 8 up, 8 in 2024-02-05T07:58:10.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:09 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:10.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:10.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:10.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-61"}]: dispatch 2024-02-05T07:58:10.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-61"}]: dispatch 2024-02-05T07:58:10.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:10 smithi157 ceph-mon[33345]: pgmap v549: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:58:10.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:10 smithi157 ceph-mon[33345]: osdmap e389: 8 total, 8 up, 8 in 2024-02-05T07:58:11.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:10 smithi131 ceph-mon[41654]: pgmap v549: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:58:11.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:10 smithi131 ceph-mon[41654]: osdmap e389: 8 total, 8 up, 8 in 2024-02-05T07:58:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:10 smithi131 ceph-mon[29507]: pgmap v549: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:58:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:10 smithi131 ceph-mon[29507]: osdmap e389: 8 total, 8 up, 8 in 2024-02-05T07:58:11.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:11 smithi157 ceph-mon[33345]: osdmap e390: 8 total, 8 up, 8 in 2024-02-05T07:58:11.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:11.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:12.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:11 smithi131 ceph-mon[41654]: osdmap e390: 8 total, 8 up, 8 in 2024-02-05T07:58:12.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:12.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:12.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:11 smithi131 ceph-mon[29507]: osdmap e390: 8 total, 8 up, 8 in 2024-02-05T07:58:12.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:12.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:12.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:12 smithi157 ceph-mon[33345]: pgmap v552: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:58:12.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:58:12.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:12 smithi157 ceph-mon[33345]: osdmap e391: 8 total, 8 up, 8 in 2024-02-05T07:58:12.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:58:12.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:12.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:12.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:12 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:58:13.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:58:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:58:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:58:13.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:12 smithi131 ceph-mon[41654]: pgmap v552: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:58:13.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:58:13.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:12 smithi131 ceph-mon[41654]: osdmap e391: 8 total, 8 up, 8 in 2024-02-05T07:58:13.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:58:13.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:13.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:13.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:12 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:58:13.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:12 smithi131 ceph-mon[29507]: pgmap v552: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:58:13.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:58:13.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:12 smithi131 ceph-mon[29507]: osdmap e391: 8 total, 8 up, 8 in 2024-02-05T07:58:13.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:58:13.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:13.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:13.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:12 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:58:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:13 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:58:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:13 smithi131 ceph-mon[41654]: osdmap e392: 8 total, 8 up, 8 in 2024-02-05T07:58:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:13 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:58:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:13 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:58:14.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:13 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:58:14.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:13 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:58:14.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:13 smithi131 ceph-mon[29507]: osdmap e392: 8 total, 8 up, 8 in 2024-02-05T07:58:14.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:13 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:58:14.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:13 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:58:14.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:13 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:58:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:13 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:58:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:13 smithi157 ceph-mon[33345]: osdmap e392: 8 total, 8 up, 8 in 2024-02-05T07:58:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:13 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:58:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:13 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:58:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:13 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:58:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:14 smithi131 ceph-mon[41654]: pgmap v555: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:58:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:14 smithi131 ceph-mon[41654]: osdmap e393: 8 total, 8 up, 8 in 2024-02-05T07:58:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:14 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:58:15.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:14 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:15.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:14 smithi131 ceph-mon[29507]: pgmap v555: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:58:15.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:14 smithi131 ceph-mon[29507]: osdmap e393: 8 total, 8 up, 8 in 2024-02-05T07:58:15.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:14 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:58:15.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:14 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:14 smithi157 ceph-mon[33345]: pgmap v555: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:58:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:14 smithi157 ceph-mon[33345]: osdmap e393: 8 total, 8 up, 8 in 2024-02-05T07:58:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:14 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:58:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:14 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:16.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:15 smithi131 ceph-mon[41654]: osdmap e394: 8 total, 8 up, 8 in 2024-02-05T07:58:16.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:15 smithi131 ceph-mon[29507]: osdmap e394: 8 total, 8 up, 8 in 2024-02-05T07:58:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:15 smithi157 ceph-mon[33345]: osdmap e394: 8 total, 8 up, 8 in 2024-02-05T07:58:17.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:16 smithi131 ceph-mon[41654]: pgmap v558: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:58:17.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:16 smithi131 ceph-mon[41654]: osdmap e395: 8 total, 8 up, 8 in 2024-02-05T07:58:17.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:16 smithi131 ceph-mon[29507]: pgmap v558: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:58:17.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:16 smithi131 ceph-mon[29507]: osdmap e395: 8 total, 8 up, 8 in 2024-02-05T07:58:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:16 smithi157 ceph-mon[33345]: pgmap v558: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:58:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:16 smithi157 ceph-mon[33345]: osdmap e395: 8 total, 8 up, 8 in 2024-02-05T07:58:18.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:17 smithi131 ceph-mon[41654]: osdmap e396: 8 total, 8 up, 8 in 2024-02-05T07:58:18.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:17 smithi131 ceph-mon[29507]: osdmap e396: 8 total, 8 up, 8 in 2024-02-05T07:58:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:17 smithi157 ceph-mon[33345]: osdmap e396: 8 total, 8 up, 8 in 2024-02-05T07:58:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:18 smithi157 ceph-mon[33345]: pgmap v561: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:58:19.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:58:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:58:19.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:18 smithi131 ceph-mon[41654]: pgmap v561: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:58:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:18 smithi131 ceph-mon[29507]: pgmap v561: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:58:20.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:19 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:20.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:19 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:19 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:20 smithi157 ceph-mon[33345]: pgmap v562: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-05T07:58:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:20 smithi131 ceph-mon[41654]: pgmap v562: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-05T07:58:21.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:20 smithi131 ceph-mon[29507]: pgmap v562: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-05T07:58:22.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:22 smithi131 ceph-mon[41654]: pgmap v563: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 918 B/s wr, 3 op/s 2024-02-05T07:58:22.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:22 smithi131 ceph-mon[41654]: osdmap e397: 8 total, 8 up, 8 in 2024-02-05T07:58:22.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:22 smithi131 ceph-mon[29507]: pgmap v563: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 918 B/s wr, 3 op/s 2024-02-05T07:58:22.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:22 smithi131 ceph-mon[29507]: osdmap e397: 8 total, 8 up, 8 in 2024-02-05T07:58:22.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:22 smithi157 ceph-mon[33345]: pgmap v563: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 918 B/s wr, 3 op/s 2024-02-05T07:58:22.973 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:22 smithi157 ceph-mon[33345]: osdmap e397: 8 total, 8 up, 8 in 2024-02-05T07:58:23.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:58:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:58:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:58:24.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:24 smithi157 ceph-mon[33345]: pgmap v565: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 800 B/s wr, 3 op/s 2024-02-05T07:58:25.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:24 smithi131 ceph-mon[41654]: pgmap v565: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 800 B/s wr, 3 op/s 2024-02-05T07:58:25.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:24 smithi131 ceph-mon[29507]: pgmap v565: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 800 B/s wr, 3 op/s 2024-02-05T07:58:26.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:26 smithi157 ceph-mon[33345]: pgmap v566: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 708 B/s wr, 3 op/s 2024-02-05T07:58:27.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:26 smithi131 ceph-mon[41654]: pgmap v566: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 708 B/s wr, 3 op/s 2024-02-05T07:58:27.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:26 smithi131 ceph-mon[29507]: pgmap v566: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 708 B/s wr, 3 op/s 2024-02-05T07:58:28.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:28 smithi157 ceph-mon[33345]: pgmap v567: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 614 B/s wr, 2 op/s 2024-02-05T07:58:28.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:28 smithi157 ceph-mon[33345]: osdmap e398: 8 total, 8 up, 8 in 2024-02-05T07:58:28.972 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:58:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:58:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:28 smithi131 ceph-mon[41654]: pgmap v567: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 614 B/s wr, 2 op/s 2024-02-05T07:58:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:28 smithi131 ceph-mon[41654]: osdmap e398: 8 total, 8 up, 8 in 2024-02-05T07:58:29.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:28 smithi131 ceph-mon[29507]: pgmap v567: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 614 B/s wr, 2 op/s 2024-02-05T07:58:29.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:28 smithi131 ceph-mon[29507]: osdmap e398: 8 total, 8 up, 8 in 2024-02-05T07:58:29.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:29 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:29.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:29 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:30.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:29 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:30.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:29 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:30.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:29 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:30.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:29 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:30.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:30 smithi157 ceph-mon[33345]: pgmap v569: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:58:31.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:30 smithi131 ceph-mon[41654]: pgmap v569: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:58:31.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:30 smithi131 ceph-mon[29507]: pgmap v569: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:58:32.942 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:32 smithi131 ceph-mon[41654]: pgmap v570: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:58:32.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:32 smithi131 ceph-mon[41654]: osdmap e399: 8 total, 8 up, 8 in 2024-02-05T07:58:32.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:32 smithi131 ceph-mon[29507]: pgmap v570: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:58:32.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:32 smithi131 ceph-mon[29507]: osdmap e399: 8 total, 8 up, 8 in 2024-02-05T07:58:32.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:32 smithi157 ceph-mon[33345]: pgmap v570: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:58:32.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:32 smithi157 ceph-mon[33345]: osdmap e399: 8 total, 8 up, 8 in 2024-02-05T07:58:33.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:58:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:58:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:58:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:34 smithi157 ceph-mon[33345]: pgmap v572: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:58:35.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:34 smithi131 ceph-mon[41654]: pgmap v572: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:58:35.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:34 smithi131 ceph-mon[29507]: pgmap v572: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:58:36.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:36 smithi157 ceph-mon[33345]: pgmap v573: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:58:37.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:36 smithi131 ceph-mon[41654]: pgmap v573: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:58:37.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:36 smithi131 ceph-mon[29507]: pgmap v573: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:58:38.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:38 smithi157 ceph-mon[33345]: pgmap v574: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:58:38.972 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:58:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:58:39.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:38 smithi131 ceph-mon[41654]: pgmap v574: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:58:39.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:38 smithi131 ceph-mon[29507]: pgmap v574: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:58:39.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:39 smithi157 ceph-mon[33345]: osdmap e400: 8 total, 8 up, 8 in 2024-02-05T07:58:39.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:39 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:40.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:39 smithi131 ceph-mon[41654]: osdmap e400: 8 total, 8 up, 8 in 2024-02-05T07:58:40.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:39 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:40.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:39 smithi131 ceph-mon[29507]: osdmap e400: 8 total, 8 up, 8 in 2024-02-05T07:58:40.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:39 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:40.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:40 smithi157 ceph-mon[33345]: pgmap v576: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:58:41.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:40 smithi131 ceph-mon[41654]: pgmap v576: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:58:41.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:40 smithi131 ceph-mon[29507]: pgmap v576: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:58:42.942 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:42 smithi131 ceph-mon[41654]: pgmap v577: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:58:42.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:42 smithi131 ceph-mon[29507]: pgmap v577: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:58:42.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:42 smithi157 ceph-mon[33345]: pgmap v577: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:58:43.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:58:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:58:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:58:44.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:44 smithi157 ceph-mon[33345]: pgmap v578: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:58:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:44 smithi131 ceph-mon[41654]: pgmap v578: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:58:45.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:44 smithi131 ceph-mon[29507]: pgmap v578: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:58:45.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:45 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:46.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:45 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:46.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:45 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:46.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:46 smithi157 ceph-mon[33345]: pgmap v579: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:58:46.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:46 smithi157 ceph-mon[33345]: osdmap e401: 8 total, 8 up, 8 in 2024-02-05T07:58:47.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:46 smithi131 ceph-mon[41654]: pgmap v579: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:58:47.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:46 smithi131 ceph-mon[41654]: osdmap e401: 8 total, 8 up, 8 in 2024-02-05T07:58:47.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:46 smithi131 ceph-mon[29507]: pgmap v579: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:58:47.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:46 smithi131 ceph-mon[29507]: osdmap e401: 8 total, 8 up, 8 in 2024-02-05T07:58:48.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:48 smithi157 ceph-mon[33345]: pgmap v581: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 579 B/s rd, 0 op/s 2024-02-05T07:58:48.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:58:48 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:58:49.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:48 smithi131 ceph-mon[41654]: pgmap v581: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 579 B/s rd, 0 op/s 2024-02-05T07:58:49.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:48 smithi131 ceph-mon[29507]: pgmap v581: 292 pgs: 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 579 B/s rd, 0 op/s 2024-02-05T07:58:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-63"}]: dispatch 2024-02-05T07:58:49.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-63"}]: dispatch 2024-02-05T07:58:49.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:49 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-63"}]: dispatch 2024-02-05T07:58:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-63"}]: dispatch 2024-02-05T07:58:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:49 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:50.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:50.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:58:50.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-63"}]: dispatch 2024-02-05T07:58:50.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-63"}]: dispatch 2024-02-05T07:58:50.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:49 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:50.919 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:50 smithi131 ceph-mon[41654]: pgmap v582: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:58:50.919 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:50 smithi131 ceph-mon[41654]: osdmap e402: 8 total, 8 up, 8 in 2024-02-05T07:58:50.920 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:50 smithi131 ceph-mon[29507]: pgmap v582: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:58:50.920 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:50 smithi131 ceph-mon[29507]: osdmap e402: 8 total, 8 up, 8 in 2024-02-05T07:58:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:50 smithi157 ceph-mon[33345]: pgmap v582: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:58:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:50 smithi157 ceph-mon[33345]: osdmap e402: 8 total, 8 up, 8 in 2024-02-05T07:58:51.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:51 smithi157 ceph-mon[33345]: osdmap e403: 8 total, 8 up, 8 in 2024-02-05T07:58:51.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:51.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:51 smithi131 ceph-mon[41654]: osdmap e403: 8 total, 8 up, 8 in 2024-02-05T07:58:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:52.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:51 smithi131 ceph-mon[29507]: osdmap e403: 8 total, 8 up, 8 in 2024-02-05T07:58:52.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:52.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:58:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:52 smithi131 ceph-mon[41654]: pgmap v585: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:58:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:52 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:58:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:58:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:52 smithi131 ceph-mon[41654]: osdmap e404: 8 total, 8 up, 8 in 2024-02-05T07:58:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:58:52.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:52.944 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:52.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:52 smithi131 ceph-mon[29507]: pgmap v585: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:58:52.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:52 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:58:52.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:58:52.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:52 smithi131 ceph-mon[29507]: osdmap e404: 8 total, 8 up, 8 in 2024-02-05T07:58:52.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:58:52.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:52.944 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:52 smithi157 ceph-mon[33345]: pgmap v585: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:58:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:52 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:58:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:58:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:52 smithi157 ceph-mon[33345]: osdmap e404: 8 total, 8 up, 8 in 2024-02-05T07:58:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:58:52.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:52.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:58:53.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:58:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:58:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:58:53.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:58:53.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:53 smithi157 ceph-mon[33345]: osdmap e405: 8 total, 8 up, 8 in 2024-02-05T07:58:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:58:54.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:53 smithi131 ceph-mon[41654]: osdmap e405: 8 total, 8 up, 8 in 2024-02-05T07:58:54.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:58:54.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:53 smithi131 ceph-mon[29507]: osdmap e405: 8 total, 8 up, 8 in 2024-02-05T07:58:54.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:54 smithi157 ceph-mon[33345]: pgmap v588: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:58:54.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:54 smithi157 ceph-mon[33345]: osdmap e406: 8 total, 8 up, 8 in 2024-02-05T07:58:55.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:54 smithi131 ceph-mon[41654]: pgmap v588: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:58:55.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:54 smithi131 ceph-mon[41654]: osdmap e406: 8 total, 8 up, 8 in 2024-02-05T07:58:55.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:54 smithi131 ceph-mon[29507]: pgmap v588: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:58:55.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:54 smithi131 ceph-mon[29507]: osdmap e406: 8 total, 8 up, 8 in 2024-02-05T07:58:55.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:55 smithi157 ceph-mon[33345]: osdmap e407: 8 total, 8 up, 8 in 2024-02-05T07:58:56.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:55 smithi131 ceph-mon[41654]: osdmap e407: 8 total, 8 up, 8 in 2024-02-05T07:58:56.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:55 smithi131 ceph-mon[29507]: osdmap e407: 8 total, 8 up, 8 in 2024-02-05T07:58:56.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:56 smithi157 ceph-mon[33345]: pgmap v591: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:58:56.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:56 smithi157 ceph-mon[33345]: osdmap e408: 8 total, 8 up, 8 in 2024-02-05T07:58:57.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:56 smithi131 ceph-mon[41654]: pgmap v591: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:58:57.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:56 smithi131 ceph-mon[41654]: osdmap e408: 8 total, 8 up, 8 in 2024-02-05T07:58:57.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:56 smithi131 ceph-mon[29507]: pgmap v591: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:58:57.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:56 smithi131 ceph-mon[29507]: osdmap e408: 8 total, 8 up, 8 in 2024-02-05T07:58:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:58 smithi157 ceph-mon[33345]: pgmap v593: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-05T07:58:58.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:58:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:58:59.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:58 smithi131 ceph-mon[29507]: pgmap v593: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-05T07:58:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:58 smithi131 ceph-mon[41654]: pgmap v593: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-05T07:58:59.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:59 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:59.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:58:59 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:00.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:59 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:00.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:58:59 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:59 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:00.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:58:59 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:00.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:00 smithi157 ceph-mon[33345]: pgmap v594: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-02-05T07:59:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:00 smithi131 ceph-mon[41654]: pgmap v594: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-02-05T07:59:01.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:00 smithi131 ceph-mon[29507]: pgmap v594: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-02-05T07:59:01.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:01 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:59:01.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:01 smithi157 ceph-mon[33345]: osdmap e409: 8 total, 8 up, 8 in 2024-02-05T07:59:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:01 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:59:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:01 smithi131 ceph-mon[41654]: osdmap e409: 8 total, 8 up, 8 in 2024-02-05T07:59:02.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:01 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:59:02.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:01 smithi131 ceph-mon[29507]: osdmap e409: 8 total, 8 up, 8 in 2024-02-05T07:59:02.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:02 smithi131 ceph-mon[29507]: pgmap v595: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 525 B/s wr, 3 op/s 2024-02-05T07:59:02.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:02 smithi131 ceph-mon[41654]: pgmap v595: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 525 B/s wr, 3 op/s 2024-02-05T07:59:02.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:02 smithi157 ceph-mon[33345]: pgmap v595: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 525 B/s wr, 3 op/s 2024-02-05T07:59:03.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:59:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:59:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:59:04.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:04 smithi157 ceph-mon[33345]: pgmap v597: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T07:59:05.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:04 smithi131 ceph-mon[41654]: pgmap v597: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T07:59:05.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:04 smithi131 ceph-mon[29507]: pgmap v597: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T07:59:06.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:06 smithi157 ceph-mon[33345]: pgmap v598: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 418 B/s wr, 3 op/s 2024-02-05T07:59:06.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-65"}]: dispatch 2024-02-05T07:59:06.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-65"}]: dispatch 2024-02-05T07:59:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:06 smithi131 ceph-mon[41654]: pgmap v598: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 418 B/s wr, 3 op/s 2024-02-05T07:59:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-65"}]: dispatch 2024-02-05T07:59:07.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-65"}]: dispatch 2024-02-05T07:59:07.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:06 smithi131 ceph-mon[29507]: pgmap v598: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 418 B/s wr, 3 op/s 2024-02-05T07:59:07.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:07.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:07.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-65"}]: dispatch 2024-02-05T07:59:07.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-65"}]: dispatch 2024-02-05T07:59:07.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:07 smithi157 ceph-mon[33345]: osdmap e410: 8 total, 8 up, 8 in 2024-02-05T07:59:08.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:07 smithi131 ceph-mon[41654]: osdmap e410: 8 total, 8 up, 8 in 2024-02-05T07:59:08.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:07 smithi131 ceph-mon[29507]: osdmap e410: 8 total, 8 up, 8 in 2024-02-05T07:59:08.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103aee0 2024-02-05T07:59:08.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103b4b0 2024-02-05T07:59:08.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103a9b0 2024-02-05T07:59:08.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103aae0 2024-02-05T07:59:08.699 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103abf0 2024-02-05T07:59:08.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103b600 2024-02-05T07:59:08.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103b010 2024-02-05T07:59:08.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103b160 2024-02-05T07:59:08.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103b290 2024-02-05T07:59:08.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103b730 2024-02-05T07:59:08.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103bbc0 2024-02-05T07:59:08.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103bcf0 2024-02-05T07:59:08.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510395c0 2024-02-05T07:59:08.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251039710 2024-02-05T07:59:08.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103c2c0 2024-02-05T07:59:08.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103be20 2024-02-05T07:59:08.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103bf50 2024-02-05T07:59:08.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103c080 2024-02-05T07:59:08.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103c1b0 2024-02-05T07:59:08.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103c890 2024-02-05T07:59:08.700 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103c3f0 2024-02-05T07:59:08.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103c500 2024-02-05T07:59:08.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103c630 2024-02-05T07:59:08.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103c780 2024-02-05T07:59:08.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103ce40 2024-02-05T07:59:08.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103c9a0 2024-02-05T07:59:08.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103caf0 2024-02-05T07:59:08.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103cc00 2024-02-05T07:59:08.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103cf90 2024-02-05T07:59:08.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103d440 2024-02-05T07:59:08.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103d590 2024-02-05T07:59:08.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103d0c0 2024-02-05T07:59:08.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103d210 2024-02-05T07:59:08.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103db60 2024-02-05T07:59:08.701 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103d6c0 2024-02-05T07:59:08.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103d810 2024-02-05T07:59:08.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103d920 2024-02-05T07:59:08.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103da30 2024-02-05T07:59:08.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103e150 2024-02-05T07:59:08.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103dcb0 2024-02-05T07:59:08.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103dde0 2024-02-05T07:59:08.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103df10 2024-02-05T07:59:08.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103e040 2024-02-05T07:59:08.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103e720 2024-02-05T07:59:08.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103e280 2024-02-05T07:59:08.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103e3d0 2024-02-05T07:59:08.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103e500 2024-02-05T07:59:08.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103e850 2024-02-05T07:59:08.702 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103ece0 2024-02-05T07:59:08.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103ee10 2024-02-05T07:59:08.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103f3e0 2024-02-05T07:59:08.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103e980 2024-02-05T07:59:08.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103eab0 2024-02-05T07:59:08.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103ef40 2024-02-05T07:59:08.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103f070 2024-02-05T07:59:08.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103f1a0 2024-02-05T07:59:08.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103f2d0 2024-02-05T07:59:08.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103f9b0 2024-02-05T07:59:08.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103f510 2024-02-05T07:59:08.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103f660 2024-02-05T07:59:08.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103f790 2024-02-05T07:59:08.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103fae0 2024-02-05T07:59:08.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103ff70 2024-02-05T07:59:08.703 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510400a0 2024-02-05T07:59:08.704 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103e610 2024-02-05T07:59:08.704 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510401d0 2024-02-05T07:59:08.704 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103b860 2024-02-05T07:59:08.704 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103b990 2024-02-05T07:59:08.704 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103baa0 2024-02-05T07:59:08.704 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510407a0 2024-02-05T07:59:08.704 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251040300 2024-02-05T07:59:08.704 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251040430 2024-02-05T07:59:08.704 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251040540 2024-02-05T07:59:08.704 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251040690 2024-02-05T07:59:08.704 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251040d50 2024-02-05T07:59:08.704 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510408b0 2024-02-05T07:59:08.704 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251040a00 2024-02-05T07:59:08.704 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251040b30 2024-02-05T07:59:08.705 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251040ea0 2024-02-05T07:59:08.705 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251041330 2024-02-05T07:59:08.705 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251041460 2024-02-05T07:59:08.705 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251041a30 2024-02-05T07:59:08.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:59:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:59:08.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:08 smithi157 ceph-mon[33345]: pgmap v600: 260 pgs: 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:08 smithi157 ceph-mon[33345]: osdmap e411: 8 total, 8 up, 8 in 2024-02-05T07:59:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:08.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:08 smithi131 ceph-mon[41654]: pgmap v600: 260 pgs: 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:08 smithi131 ceph-mon[41654]: osdmap e411: 8 total, 8 up, 8 in 2024-02-05T07:59:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:08 smithi131 ceph-mon[29507]: pgmap v600: 260 pgs: 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:08 smithi131 ceph-mon[29507]: osdmap e411: 8 total, 8 up, 8 in 2024-02-05T07:59:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:09.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:09.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:59:09.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:09 smithi157 ceph-mon[33345]: osdmap e412: 8 total, 8 up, 8 in 2024-02-05T07:59:09.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:59:09.972 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:09 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:59:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:09 smithi131 ceph-mon[41654]: osdmap e412: 8 total, 8 up, 8 in 2024-02-05T07:59:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:59:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:09 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:59:10.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:09 smithi131 ceph-mon[29507]: osdmap e412: 8 total, 8 up, 8 in 2024-02-05T07:59:10.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:59:10.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:09 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:10.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:10 smithi157 ceph-mon[33345]: pgmap v603: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:10.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:10 smithi157 ceph-mon[33345]: osdmap e413: 8 total, 8 up, 8 in 2024-02-05T07:59:10.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:10.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:10.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-67"}]: dispatch 2024-02-05T07:59:10.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-67"}]: dispatch 2024-02-05T07:59:11.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:10 smithi131 ceph-mon[41654]: pgmap v603: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:11.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:10 smithi131 ceph-mon[41654]: osdmap e413: 8 total, 8 up, 8 in 2024-02-05T07:59:11.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:11.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:11.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-67"}]: dispatch 2024-02-05T07:59:11.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-67"}]: dispatch 2024-02-05T07:59:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:10 smithi131 ceph-mon[29507]: pgmap v603: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:10 smithi131 ceph-mon[29507]: osdmap e413: 8 total, 8 up, 8 in 2024-02-05T07:59:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:11.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:11.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-67"}]: dispatch 2024-02-05T07:59:11.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-67"}]: dispatch 2024-02-05T07:59:12.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:11 smithi131 ceph-mon[41654]: osdmap e414: 8 total, 8 up, 8 in 2024-02-05T07:59:12.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:11 smithi131 ceph-mon[29507]: osdmap e414: 8 total, 8 up, 8 in 2024-02-05T07:59:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:11 smithi157 ceph-mon[33345]: osdmap e414: 8 total, 8 up, 8 in 2024-02-05T07:59:13.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:59:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:59:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:59:13.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:12 smithi131 ceph-mon[41654]: pgmap v606: 260 pgs: 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:13.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:12 smithi131 ceph-mon[41654]: osdmap e415: 8 total, 8 up, 8 in 2024-02-05T07:59:13.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:13.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:13.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:12 smithi131 ceph-mon[29507]: pgmap v606: 260 pgs: 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:13.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:12 smithi131 ceph-mon[29507]: osdmap e415: 8 total, 8 up, 8 in 2024-02-05T07:59:13.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:13.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:12 smithi157 ceph-mon[33345]: pgmap v606: 260 pgs: 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:13.260 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:12 smithi157 ceph-mon[33345]: osdmap e415: 8 total, 8 up, 8 in 2024-02-05T07:59:13.260 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:13.260 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:14.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:59:14.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[41654]: osdmap e416: 8 total, 8 up, 8 in 2024-02-05T07:59:14.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:59:14.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:14.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:14.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-69"}]: dispatch 2024-02-05T07:59:14.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-69"}]: dispatch 2024-02-05T07:59:14.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:59:14.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[41654]: osdmap e417: 8 total, 8 up, 8 in 2024-02-05T07:59:14.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:59:14.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[29507]: osdmap e416: 8 total, 8 up, 8 in 2024-02-05T07:59:14.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:59:14.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:14.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:14.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-69"}]: dispatch 2024-02-05T07:59:14.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-69"}]: dispatch 2024-02-05T07:59:14.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:59:14.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:13 smithi131 ceph-mon[29507]: osdmap e417: 8 total, 8 up, 8 in 2024-02-05T07:59:14.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:13 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:59:14.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:13 smithi157 ceph-mon[33345]: osdmap e416: 8 total, 8 up, 8 in 2024-02-05T07:59:14.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:13 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:59:14.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:13 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:14.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:13 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:14.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:13 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-69"}]: dispatch 2024-02-05T07:59:14.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:13 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-69"}]: dispatch 2024-02-05T07:59:14.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:13 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:59:14.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:13 smithi157 ceph-mon[33345]: osdmap e417: 8 total, 8 up, 8 in 2024-02-05T07:59:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:14 smithi131 ceph-mon[41654]: pgmap v609: 292 pgs: 292 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T07:59:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:14 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:59:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:14 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:59:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:14 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:59:15.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:14 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:15.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:14 smithi131 ceph-mon[41654]: osdmap e418: 8 total, 8 up, 8 in 2024-02-05T07:59:15.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:15.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:15.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:14 smithi131 ceph-mon[29507]: pgmap v609: 292 pgs: 292 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T07:59:15.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:14 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:59:15.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:14 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:59:15.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:14 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:59:15.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:14 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:15.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:14 smithi131 ceph-mon[29507]: osdmap e418: 8 total, 8 up, 8 in 2024-02-05T07:59:15.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:15.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:14 smithi157 ceph-mon[33345]: pgmap v609: 292 pgs: 292 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T07:59:15.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:14 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:59:15.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:14 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:59:15.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:14 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T07:59:15.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:14 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:15.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:14 smithi157 ceph-mon[33345]: osdmap e418: 8 total, 8 up, 8 in 2024-02-05T07:59:15.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:15.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:17.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:16 smithi131 ceph-mon[41654]: pgmap v612: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:59:17.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:59:17.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:16 smithi131 ceph-mon[41654]: osdmap e419: 8 total, 8 up, 8 in 2024-02-05T07:59:17.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:59:17.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:59:17.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:59:17.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:16 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:59:17.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:16 smithi131 ceph-mon[29507]: pgmap v612: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:59:17.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:59:17.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:16 smithi131 ceph-mon[29507]: osdmap e419: 8 total, 8 up, 8 in 2024-02-05T07:59:17.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:59:17.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:59:17.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:59:17.020 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:16 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:59:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:16 smithi157 ceph-mon[33345]: pgmap v612: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:59:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:59:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:16 smithi157 ceph-mon[33345]: osdmap e419: 8 total, 8 up, 8 in 2024-02-05T07:59:17.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:59:17.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:59:17.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:59:17.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:16 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:59:18.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:59:18.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:17 smithi131 ceph-mon[41654]: osdmap e420: 8 total, 8 up, 8 in 2024-02-05T07:59:18.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:59:18.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:17 smithi131 ceph-mon[29507]: osdmap e420: 8 total, 8 up, 8 in 2024-02-05T07:59:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:59:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:17 smithi157 ceph-mon[33345]: osdmap e420: 8 total, 8 up, 8 in 2024-02-05T07:59:19.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:18 smithi131 ceph-mon[41654]: pgmap v615: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:19.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:18 smithi131 ceph-mon[41654]: osdmap e421: 8 total, 8 up, 8 in 2024-02-05T07:59:19.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:18 smithi131 ceph-mon[29507]: pgmap v615: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:19.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:18 smithi131 ceph-mon[29507]: osdmap e421: 8 total, 8 up, 8 in 2024-02-05T07:59:19.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:59:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:59:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:18 smithi157 ceph-mon[33345]: pgmap v615: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:18 smithi157 ceph-mon[33345]: osdmap e421: 8 total, 8 up, 8 in 2024-02-05T07:59:20.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:19 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:20.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:19 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:19 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:21.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:20 smithi131 ceph-mon[41654]: pgmap v617: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-02-05T07:59:21.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:20 smithi131 ceph-mon[29507]: pgmap v617: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-02-05T07:59:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:20 smithi157 ceph-mon[33345]: pgmap v617: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-02-05T07:59:22.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:21 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:59:22.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:21 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:59:22.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:21 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:59:23.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:22 smithi131 ceph-mon[29507]: pgmap v618: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-05T07:59:23.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:59:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:59:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:59:23.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:22 smithi131 ceph-mon[41654]: pgmap v618: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-05T07:59:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:22 smithi157 ceph-mon[33345]: pgmap v618: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-05T07:59:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:24 smithi157 ceph-mon[33345]: pgmap v619: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:59:25.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:24 smithi131 ceph-mon[41654]: pgmap v619: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:59:25.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:24 smithi131 ceph-mon[29507]: pgmap v619: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:59:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:26 smithi157 ceph-mon[33345]: pgmap v620: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:59:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:26 smithi131 ceph-mon[41654]: pgmap v620: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:59:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:26 smithi131 ceph-mon[29507]: pgmap v620: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:59:28.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]}]: dispatch 2024-02-05T07:59:28.123 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:27 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]}]: dispatch 2024-02-05T07:59:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]}]: dispatch 2024-02-05T07:59:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:27 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]}]: dispatch 2024-02-05T07:59:28.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]}]: dispatch 2024-02-05T07:59:28.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:27 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]}]: dispatch 2024-02-05T07:59:29.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:59:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:59:29.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:28 smithi157 ceph-mon[33345]: pgmap v621: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-05T07:59:29.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:28 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]}]': finished 2024-02-05T07:59:29.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:28 smithi157 ceph-mon[33345]: osdmap e422: 8 total, 8 up, 8 in 2024-02-05T07:59:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:28 smithi131 ceph-mon[41654]: pgmap v621: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-05T07:59:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:28 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]}]': finished 2024-02-05T07:59:29.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:28 smithi131 ceph-mon[41654]: osdmap e422: 8 total, 8 up, 8 in 2024-02-05T07:59:29.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:28 smithi131 ceph-mon[29507]: pgmap v621: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-05T07:59:29.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:28 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]}]': finished 2024-02-05T07:59:29.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:28 smithi131 ceph-mon[29507]: osdmap e422: 8 total, 8 up, 8 in 2024-02-05T07:59:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:29 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:29 smithi157 ceph-mon[33345]: osdmap e423: 8 total, 8 up, 8 in 2024-02-05T07:59:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:29 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:29 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:29 smithi131 ceph-mon[41654]: osdmap e423: 8 total, 8 up, 8 in 2024-02-05T07:59:30.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:29 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:30.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:29 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:30.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:29 smithi131 ceph-mon[29507]: osdmap e423: 8 total, 8 up, 8 in 2024-02-05T07:59:30.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:29 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:30 smithi157 ceph-mon[33345]: pgmap v624: 292 pgs: 1 peering, 291 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 383 B/s wr, 3 op/s 2024-02-05T07:59:31.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:30 smithi131 ceph-mon[41654]: pgmap v624: 292 pgs: 1 peering, 291 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 383 B/s wr, 3 op/s 2024-02-05T07:59:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:30 smithi131 ceph-mon[29507]: pgmap v624: 292 pgs: 1 peering, 291 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 383 B/s wr, 3 op/s 2024-02-05T07:59:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:32 smithi157 ceph-mon[33345]: pgmap v625: 292 pgs: 1 peering, 291 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 383 B/s wr, 2 op/s 2024-02-05T07:59:33.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:59:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:59:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:59:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:32 smithi131 ceph-mon[41654]: pgmap v625: 292 pgs: 1 peering, 291 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 383 B/s wr, 2 op/s 2024-02-05T07:59:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:32 smithi131 ceph-mon[29507]: pgmap v625: 292 pgs: 1 peering, 291 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 383 B/s wr, 2 op/s 2024-02-05T07:59:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:34 smithi157 ceph-mon[33345]: pgmap v626: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 383 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-02-05T07:59:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:34 smithi131 ceph-mon[29507]: pgmap v626: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 383 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-02-05T07:59:35.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:34 smithi131 ceph-mon[41654]: pgmap v626: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 383 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-02-05T07:59:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:36 smithi157 ceph-mon[33345]: pgmap v627: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 383 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-02-05T07:59:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:36 smithi131 ceph-mon[41654]: pgmap v627: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 383 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-02-05T07:59:37.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:36 smithi131 ceph-mon[29507]: pgmap v627: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 383 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-02-05T07:59:39.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:59:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:59:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:38 smithi157 ceph-mon[33345]: pgmap v628: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 317 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-02-05T07:59:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:38 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:38 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-71"}]: dispatch 2024-02-05T07:59:39.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-71"}]: dispatch 2024-02-05T07:59:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:38 smithi131 ceph-mon[41654]: pgmap v628: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 317 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-02-05T07:59:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:38 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:38 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-71"}]: dispatch 2024-02-05T07:59:39.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-71"}]: dispatch 2024-02-05T07:59:39.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:38 smithi131 ceph-mon[29507]: pgmap v628: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 317 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-02-05T07:59:39.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:38 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:39.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:39.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:38 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-71"}]: dispatch 2024-02-05T07:59:39.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-71"}]: dispatch 2024-02-05T07:59:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:39 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:39 smithi157 ceph-mon[33345]: osdmap e424: 8 total, 8 up, 8 in 2024-02-05T07:59:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:39 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:40.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:39 smithi131 ceph-mon[41654]: osdmap e424: 8 total, 8 up, 8 in 2024-02-05T07:59:40.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:39 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:40.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:39 smithi131 ceph-mon[29507]: osdmap e424: 8 total, 8 up, 8 in 2024-02-05T07:59:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:40 smithi157 ceph-mon[33345]: pgmap v630: 260 pgs: 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:40 smithi157 ceph-mon[33345]: osdmap e425: 8 total, 8 up, 8 in 2024-02-05T07:59:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:40 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:41.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:40 smithi131 ceph-mon[41654]: pgmap v630: 260 pgs: 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:40 smithi131 ceph-mon[41654]: osdmap e425: 8 total, 8 up, 8 in 2024-02-05T07:59:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:40 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:41.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:41.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:40 smithi131 ceph-mon[29507]: pgmap v630: 260 pgs: 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:41.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:40 smithi131 ceph-mon[29507]: osdmap e425: 8 total, 8 up, 8 in 2024-02-05T07:59:41.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:40 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:41.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:59:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:41 smithi157 ceph-mon[33345]: osdmap e426: 8 total, 8 up, 8 in 2024-02-05T07:59:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:59:42.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:59:42.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:41 smithi131 ceph-mon[41654]: osdmap e426: 8 total, 8 up, 8 in 2024-02-05T07:59:42.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:59:42.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:59:42.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:41 smithi131 ceph-mon[29507]: osdmap e426: 8 total, 8 up, 8 in 2024-02-05T07:59:42.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:59:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:42 smithi157 ceph-mon[33345]: pgmap v633: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:42 smithi157 ceph-mon[33345]: osdmap e427: 8 total, 8 up, 8 in 2024-02-05T07:59:43.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:59:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:59:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:59:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:42 smithi131 ceph-mon[41654]: pgmap v633: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:42 smithi131 ceph-mon[41654]: osdmap e427: 8 total, 8 up, 8 in 2024-02-05T07:59:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:42 smithi131 ceph-mon[29507]: pgmap v633: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:42 smithi131 ceph-mon[29507]: osdmap e427: 8 total, 8 up, 8 in 2024-02-05T07:59:44.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:43 smithi157 ceph-mon[33345]: osdmap e428: 8 total, 8 up, 8 in 2024-02-05T07:59:44.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:43 smithi131 ceph-mon[41654]: osdmap e428: 8 total, 8 up, 8 in 2024-02-05T07:59:44.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:43 smithi131 ceph-mon[29507]: osdmap e428: 8 total, 8 up, 8 in 2024-02-05T07:59:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:44 smithi157 ceph-mon[33345]: pgmap v636: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-05T07:59:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:44 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:44 smithi131 ceph-mon[41654]: pgmap v636: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-05T07:59:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:44 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:45.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:44 smithi131 ceph-mon[29507]: pgmap v636: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-05T07:59:45.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:44 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:46 smithi157 ceph-mon[33345]: pgmap v637: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-05T07:59:47.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:46 smithi131 ceph-mon[41654]: pgmap v637: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-05T07:59:47.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:46 smithi131 ceph-mon[29507]: pgmap v637: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-05T07:59:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:48 smithi157 ceph-mon[33345]: pgmap v638: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-05T07:59:49.222 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:59:48 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:59:49.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:48 smithi131 ceph-mon[41654]: pgmap v638: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-05T07:59:49.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:48 smithi131 ceph-mon[29507]: pgmap v638: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-05T07:59:50.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:49 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:50.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:49 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:49 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:50 smithi157 ceph-mon[33345]: pgmap v639: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-02-05T07:59:51.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:50 smithi131 ceph-mon[41654]: pgmap v639: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-02-05T07:59:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:50 smithi131 ceph-mon[29507]: pgmap v639: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-02-05T07:59:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:52 smithi157 ceph-mon[33345]: pgmap v640: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 745 B/s wr, 3 op/s 2024-02-05T07:59:53.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:52 smithi131 ceph-mon[41654]: pgmap v640: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 745 B/s wr, 3 op/s 2024-02-05T07:59:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:52 smithi131 ceph-mon[29507]: pgmap v640: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 745 B/s wr, 3 op/s 2024-02-05T07:59:53.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 07:59:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:07:59:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:59:54.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:54.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:54.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-73"}]: dispatch 2024-02-05T07:59:54.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-73"}]: dispatch 2024-02-05T07:59:54.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:54.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:54.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-73"}]: dispatch 2024-02-05T07:59:54.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-73"}]: dispatch 2024-02-05T07:59:54.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:54.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:54.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-73"}]: dispatch 2024-02-05T07:59:54.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-73"}]: dispatch 2024-02-05T07:59:55.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:54 smithi157 ceph-mon[33345]: pgmap v641: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 675 B/s wr, 3 op/s 2024-02-05T07:59:55.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:54 smithi157 ceph-mon[33345]: osdmap e429: 8 total, 8 up, 8 in 2024-02-05T07:59:55.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:54 smithi131 ceph-mon[41654]: pgmap v641: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 675 B/s wr, 3 op/s 2024-02-05T07:59:55.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:54 smithi131 ceph-mon[41654]: osdmap e429: 8 total, 8 up, 8 in 2024-02-05T07:59:55.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:54 smithi131 ceph-mon[29507]: pgmap v641: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 675 B/s wr, 3 op/s 2024-02-05T07:59:55.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:54 smithi131 ceph-mon[29507]: osdmap e429: 8 total, 8 up, 8 in 2024-02-05T07:59:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:55 smithi157 ceph-mon[33345]: osdmap e430: 8 total, 8 up, 8 in 2024-02-05T07:59:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:55 smithi131 ceph-mon[41654]: osdmap e430: 8 total, 8 up, 8 in 2024-02-05T07:59:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:55 smithi131 ceph-mon[29507]: osdmap e430: 8 total, 8 up, 8 in 2024-02-05T07:59:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:59:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:56 smithi157 ceph-mon[33345]: pgmap v644: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T07:59:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:56 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:59:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:59:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:56 smithi157 ceph-mon[33345]: osdmap e431: 8 total, 8 up, 8 in 2024-02-05T07:59:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:59:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:56 smithi131 ceph-mon[41654]: pgmap v644: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T07:59:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:56 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:59:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:59:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:56 smithi131 ceph-mon[41654]: osdmap e431: 8 total, 8 up, 8 in 2024-02-05T07:59:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:59:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:56 smithi131 ceph-mon[29507]: pgmap v644: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T07:59:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:56 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:59:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:59:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:56 smithi131 ceph-mon[29507]: osdmap e431: 8 total, 8 up, 8 in 2024-02-05T07:59:57.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:59:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:57 smithi157 ceph-mon[33345]: osdmap e432: 8 total, 8 up, 8 in 2024-02-05T07:59:58.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:57 smithi131 ceph-mon[41654]: osdmap e432: 8 total, 8 up, 8 in 2024-02-05T07:59:58.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:57 smithi131 ceph-mon[29507]: osdmap e432: 8 total, 8 up, 8 in 2024-02-05T07:59:59.167 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 07:59:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T07:59:59.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:59 smithi157 ceph-mon[33345]: pgmap v647: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:59.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:59 smithi157 ceph-mon[33345]: osdmap e433: 8 total, 8 up, 8 in 2024-02-05T07:59:59.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:59.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:59.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-75"}]: dispatch 2024-02-05T07:59:59.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 07:59:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-75"}]: dispatch 2024-02-05T07:59:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:59 smithi131 ceph-mon[41654]: pgmap v647: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:59 smithi131 ceph-mon[41654]: osdmap e433: 8 total, 8 up, 8 in 2024-02-05T07:59:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-75"}]: dispatch 2024-02-05T07:59:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 07:59:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-75"}]: dispatch 2024-02-05T07:59:59.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:59 smithi131 ceph-mon[29507]: pgmap v647: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:59:59.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:59 smithi131 ceph-mon[29507]: osdmap e433: 8 total, 8 up, 8 in 2024-02-05T07:59:59.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:59.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T07:59:59.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-75"}]: dispatch 2024-02-05T07:59:59.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 07:59:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-75"}]: dispatch 2024-02-05T08:00:00.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:00 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:00.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:00 smithi157 ceph-mon[33345]: osdmap e434: 8 total, 8 up, 8 in 2024-02-05T08:00:00.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:00 smithi157 ceph-mon[33345]: pgmap v650: 260 pgs: 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 767 B/s wr, 181 op/s 2024-02-05T08:00:00.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:00.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:00 smithi157 ceph-mon[33345]: osdmap e435: 8 total, 8 up, 8 in 2024-02-05T08:00:00.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:00.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:00.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:00 smithi157 ceph-mon[33345]: Health detail: HEALTH_WARN 3 pool(s) do not have an application enabled 2024-02-05T08:00:00.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:00 smithi157 ceph-mon[33345]: [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-02-05T08:00:00.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:00 smithi157 ceph-mon[33345]: application not enabled on pool 'poolrace.1804289383' 2024-02-05T08:00:00.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:00 smithi157 ceph-mon[33345]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-05T08:00:00.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:00 smithi157 ceph-mon[33345]: application not enabled on pool 'poolrace2.846930886' 2024-02-05T08:00:00.473 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:00 smithi157 ceph-mon[33345]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-05T08:00:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[41654]: osdmap e434: 8 total, 8 up, 8 in 2024-02-05T08:00:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[41654]: pgmap v650: 260 pgs: 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 767 B/s wr, 181 op/s 2024-02-05T08:00:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[41654]: osdmap e435: 8 total, 8 up, 8 in 2024-02-05T08:00:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[41654]: Health detail: HEALTH_WARN 3 pool(s) do not have an application enabled 2024-02-05T08:00:00.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[41654]: [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-02-05T08:00:00.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[41654]: application not enabled on pool 'poolrace.1804289383' 2024-02-05T08:00:00.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[41654]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-05T08:00:00.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[41654]: application not enabled on pool 'poolrace2.846930886' 2024-02-05T08:00:00.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[41654]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-05T08:00:00.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:00.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[29507]: osdmap e434: 8 total, 8 up, 8 in 2024-02-05T08:00:00.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[29507]: pgmap v650: 260 pgs: 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 767 B/s wr, 181 op/s 2024-02-05T08:00:00.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:00.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[29507]: osdmap e435: 8 total, 8 up, 8 in 2024-02-05T08:00:00.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:00.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:00.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[29507]: Health detail: HEALTH_WARN 3 pool(s) do not have an application enabled 2024-02-05T08:00:00.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[29507]: [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-02-05T08:00:00.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[29507]: application not enabled on pool 'poolrace.1804289383' 2024-02-05T08:00:00.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[29507]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-05T08:00:00.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[29507]: application not enabled on pool 'poolrace2.846930886' 2024-02-05T08:00:00.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:00 smithi131 ceph-mon[29507]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-05T08:00:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:01 smithi157 ceph-mon[33345]: osdmap e436: 8 total, 8 up, 8 in 2024-02-05T08:00:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:01 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:02.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:02.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:01 smithi131 ceph-mon[29507]: osdmap e436: 8 total, 8 up, 8 in 2024-02-05T08:00:02.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:02.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:01 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:01 smithi131 ceph-mon[41654]: osdmap e436: 8 total, 8 up, 8 in 2024-02-05T08:00:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:01 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:02 smithi157 ceph-mon[33345]: pgmap v653: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 767 B/s wr, 181 op/s 2024-02-05T08:00:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:02 smithi157 ceph-mon[33345]: osdmap e437: 8 total, 8 up, 8 in 2024-02-05T08:00:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:03.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-77"}]: dispatch 2024-02-05T08:00:03.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-77"}]: dispatch 2024-02-05T08:00:03.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:00:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:00:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:00:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:02 smithi131 ceph-mon[41654]: pgmap v653: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 767 B/s wr, 181 op/s 2024-02-05T08:00:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:02 smithi131 ceph-mon[41654]: osdmap e437: 8 total, 8 up, 8 in 2024-02-05T08:00:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-77"}]: dispatch 2024-02-05T08:00:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-77"}]: dispatch 2024-02-05T08:00:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:02 smithi131 ceph-mon[29507]: pgmap v653: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 767 B/s wr, 181 op/s 2024-02-05T08:00:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:02 smithi131 ceph-mon[29507]: osdmap e437: 8 total, 8 up, 8 in 2024-02-05T08:00:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-77"}]: dispatch 2024-02-05T08:00:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-77"}]: dispatch 2024-02-05T08:00:04.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:03 smithi157 ceph-mon[33345]: osdmap e438: 8 total, 8 up, 8 in 2024-02-05T08:00:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:03 smithi131 ceph-mon[41654]: osdmap e438: 8 total, 8 up, 8 in 2024-02-05T08:00:04.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:03 smithi131 ceph-mon[29507]: osdmap e438: 8 total, 8 up, 8 in 2024-02-05T08:00:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:04 smithi157 ceph-mon[33345]: pgmap v656: 260 pgs: 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T08:00:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:04 smithi157 ceph-mon[33345]: osdmap e439: 8 total, 8 up, 8 in 2024-02-05T08:00:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:05.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:04 smithi131 ceph-mon[41654]: pgmap v656: 260 pgs: 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T08:00:05.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:04 smithi131 ceph-mon[41654]: osdmap e439: 8 total, 8 up, 8 in 2024-02-05T08:00:05.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:05.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:04 smithi131 ceph-mon[29507]: pgmap v656: 260 pgs: 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T08:00:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:04 smithi131 ceph-mon[29507]: osdmap e439: 8 total, 8 up, 8 in 2024-02-05T08:00:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:06.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:06.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:05 smithi157 ceph-mon[33345]: osdmap e440: 8 total, 8 up, 8 in 2024-02-05T08:00:06.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:06.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:06.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:05 smithi131 ceph-mon[41654]: osdmap e440: 8 total, 8 up, 8 in 2024-02-05T08:00:06.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:06.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:06.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:05 smithi131 ceph-mon[29507]: osdmap e440: 8 total, 8 up, 8 in 2024-02-05T08:00:06.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:06 smithi131 ceph-mon[41654]: pgmap v659: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T08:00:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:06 smithi131 ceph-mon[41654]: osdmap e441: 8 total, 8 up, 8 in 2024-02-05T08:00:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:06 smithi131 ceph-mon[29507]: pgmap v659: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T08:00:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:06 smithi131 ceph-mon[29507]: osdmap e441: 8 total, 8 up, 8 in 2024-02-05T08:00:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:06 smithi157 ceph-mon[33345]: pgmap v659: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T08:00:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:06 smithi157 ceph-mon[33345]: osdmap e441: 8 total, 8 up, 8 in 2024-02-05T08:00:08.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:07 smithi131 ceph-mon[41654]: osdmap e442: 8 total, 8 up, 8 in 2024-02-05T08:00:08.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:07 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.17"}]: dispatch 2024-02-05T08:00:08.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:07 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.17"}]: dispatch 2024-02-05T08:00:08.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:07 smithi131 ceph-mon[41654]: 304.17 deep-scrub starts 2024-02-05T08:00:08.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:07 smithi131 ceph-mon[41654]: 304.17 deep-scrub ok 2024-02-05T08:00:08.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:07 smithi131 ceph-mon[29507]: osdmap e442: 8 total, 8 up, 8 in 2024-02-05T08:00:08.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:07 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.17"}]: dispatch 2024-02-05T08:00:08.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:07 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.17"}]: dispatch 2024-02-05T08:00:08.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:07 smithi131 ceph-mon[29507]: 304.17 deep-scrub starts 2024-02-05T08:00:08.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:07 smithi131 ceph-mon[29507]: 304.17 deep-scrub ok 2024-02-05T08:00:08.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:07 smithi157 ceph-mon[33345]: osdmap e442: 8 total, 8 up, 8 in 2024-02-05T08:00:08.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:07 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.17"}]: dispatch 2024-02-05T08:00:08.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:07 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.17"}]: dispatch 2024-02-05T08:00:08.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:07 smithi157 ceph-mon[33345]: 304.17 deep-scrub starts 2024-02-05T08:00:08.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:07 smithi157 ceph-mon[33345]: 304.17 deep-scrub ok 2024-02-05T08:00:09.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:08 smithi157 ceph-mon[33345]: pgmap v662: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:00:09.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:00:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:00:09.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:08 smithi131 ceph-mon[41654]: pgmap v662: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:00:09.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:08 smithi131 ceph-mon[29507]: pgmap v662: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:00:10.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:09 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:09 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:10.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:09 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:10 smithi131 ceph-mon[41654]: pgmap v663: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 927 B/s rd, 927 B/s wr, 2 op/s 2024-02-05T08:00:11.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:10 smithi131 ceph-mon[29507]: pgmap v663: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 927 B/s rd, 927 B/s wr, 2 op/s 2024-02-05T08:00:11.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:10 smithi157 ceph-mon[33345]: pgmap v663: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 927 B/s rd, 927 B/s wr, 2 op/s 2024-02-05T08:00:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:12 smithi131 ceph-mon[29507]: pgmap v664: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 786 B/s rd, 786 B/s wr, 2 op/s 2024-02-05T08:00:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:12 smithi131 ceph-mon[41654]: pgmap v664: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 786 B/s rd, 786 B/s wr, 2 op/s 2024-02-05T08:00:13.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:00:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:00:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:00:13.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:12 smithi157 ceph-mon[33345]: pgmap v664: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 786 B/s rd, 786 B/s wr, 2 op/s 2024-02-05T08:00:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:15 smithi131 ceph-mon[29507]: pgmap v665: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-02-05T08:00:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:00:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:00:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:00:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:15 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:00:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:15 smithi131 ceph-mon[41654]: pgmap v665: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-02-05T08:00:15.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:00:15.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:15.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:00:15.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:00:15.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:15 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:00:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:15 smithi157 ceph-mon[33345]: pgmap v665: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-02-05T08:00:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:00:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:00:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:00:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:15 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:00:17.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:17 smithi131 ceph-mon[29507]: pgmap v666: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 539 B/s wr, 2 op/s 2024-02-05T08:00:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:17 smithi131 ceph-mon[41654]: pgmap v666: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 539 B/s wr, 2 op/s 2024-02-05T08:00:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:17 smithi157 ceph-mon[33345]: pgmap v666: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 539 B/s wr, 2 op/s 2024-02-05T08:00:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:19 smithi157 ceph-mon[33345]: pgmap v667: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 975 B/s rd, 487 B/s wr, 1 op/s 2024-02-05T08:00:19.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:00:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:00:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:19 smithi131 ceph-mon[29507]: pgmap v667: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 975 B/s rd, 487 B/s wr, 1 op/s 2024-02-05T08:00:19.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:19 smithi131 ceph-mon[41654]: pgmap v667: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 975 B/s rd, 487 B/s wr, 1 op/s 2024-02-05T08:00:20.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:20 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:20.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:20 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:20.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:20 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:21 smithi131 ceph-mon[41654]: pgmap v668: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-02-05T08:00:21.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:21 smithi131 ceph-mon[29507]: pgmap v668: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-02-05T08:00:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:21 smithi157 ceph-mon[33345]: pgmap v668: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-02-05T08:00:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:23 smithi131 ceph-mon[41654]: pgmap v669: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:23.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:00:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:00:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:00:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:23 smithi131 ceph-mon[29507]: pgmap v669: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:23.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:23 smithi157 ceph-mon[33345]: pgmap v669: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:25.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:25 smithi157 ceph-mon[33345]: pgmap v670: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:25.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:25 smithi131 ceph-mon[41654]: pgmap v670: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:25.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:25 smithi131 ceph-mon[29507]: pgmap v670: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:27 smithi131 ceph-mon[41654]: pgmap v671: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:27 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:27 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-79"}]: dispatch 2024-02-05T08:00:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-79"}]: dispatch 2024-02-05T08:00:27.269 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:00:27 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2[58548]: 2024-02-05T08:00:27.013+0000 7f974d2da700 -1 snap_mapper.add_oid found existing snaps mapped on 304:ec0c01b3:test-rados-api-smithi131-82290-80::foo:2, removing 2024-02-05T08:00:27.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:27 smithi131 ceph-mon[29507]: pgmap v671: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:27.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:27 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:27.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:27.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:27 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-79"}]: dispatch 2024-02-05T08:00:27.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-79"}]: dispatch 2024-02-05T08:00:27.471 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:00:27 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:00:27.012+0000 7fbb3733e700 -1 snap_mapper.add_oid found existing snaps mapped on 304:ec0c01b3:test-rados-api-smithi131-82290-80::foo:2, removing 2024-02-05T08:00:27.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:27 smithi157 ceph-mon[33345]: pgmap v671: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:27.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:27 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:27.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:27.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:27 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-79"}]: dispatch 2024-02-05T08:00:27.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-79"}]: dispatch 2024-02-05T08:00:27.472 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:00:27 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:00:27.012+0000 7f866451c700 -1 snap_mapper.add_oid found existing snaps mapped on 304:ec0c01b3:test-rados-api-smithi131-82290-80::foo:2, removing 2024-02-05T08:00:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:29 smithi157 ceph-mon[33345]: pgmap v672: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:29 smithi157 ceph-mon[33345]: osdmap e443: 8 total, 8 up, 8 in 2024-02-05T08:00:29.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:00:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:00:29.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:29 smithi131 ceph-mon[41654]: pgmap v672: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:29.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:29 smithi131 ceph-mon[41654]: osdmap e443: 8 total, 8 up, 8 in 2024-02-05T08:00:29.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:29 smithi131 ceph-mon[29507]: pgmap v672: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:29.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:29 smithi131 ceph-mon[29507]: osdmap e443: 8 total, 8 up, 8 in 2024-02-05T08:00:30.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:30 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:30.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:30 smithi157 ceph-mon[33345]: osdmap e444: 8 total, 8 up, 8 in 2024-02-05T08:00:30.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:30.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:30.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:30 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:30.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:30 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:30.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:30 smithi131 ceph-mon[41654]: osdmap e444: 8 total, 8 up, 8 in 2024-02-05T08:00:30.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:30.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:30.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:30 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:30.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:30 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:30.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:30 smithi131 ceph-mon[29507]: osdmap e444: 8 total, 8 up, 8 in 2024-02-05T08:00:30.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:30.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:30.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:30 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:30.717 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251040fd0 2024-02-05T08:00:30.717 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251041100 2024-02-05T08:00:30.717 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251041590 2024-02-05T08:00:30.717 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510416c0 2024-02-05T08:00:30.717 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510417f0 2024-02-05T08:00:30.718 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251041920 2024-02-05T08:00:30.718 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251041fe0 2024-02-05T08:00:30.718 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251041b40 2024-02-05T08:00:30.718 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251041c70 2024-02-05T08:00:30.718 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251041da0 2024-02-05T08:00:30.718 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251041ed0 2024-02-05T08:00:30.718 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510425b0 2024-02-05T08:00:30.718 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251042110 2024-02-05T08:00:30.718 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251042240 2024-02-05T08:00:30.718 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251042350 2024-02-05T08:00:30.718 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251042480 2024-02-05T08:00:30.718 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251042b80 2024-02-05T08:00:30.718 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510426e0 2024-02-05T08:00:30.718 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251042810 2024-02-05T08:00:30.718 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251042960 2024-02-05T08:00:30.719 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251042cb0 2024-02-05T08:00:30.719 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251043140 2024-02-05T08:00:30.719 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251043270 2024-02-05T08:00:30.719 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251042a70 2024-02-05T08:00:30.719 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510433e0 2024-02-05T08:00:30.719 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251043530 2024-02-05T08:00:30.719 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251043b20 2024-02-05T08:00:30.719 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103fc10 2024-02-05T08:00:30.719 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925103fd40 2024-02-05T08:00:30.719 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251043680 2024-02-05T08:00:30.719 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510437b0 2024-02-05T08:00:30.719 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510438e0 2024-02-05T08:00:30.719 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251043a10 2024-02-05T08:00:30.719 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510440f0 2024-02-05T08:00:30.719 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251043c50 2024-02-05T08:00:30.719 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251043d80 2024-02-05T08:00:30.720 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251043e90 2024-02-05T08:00:30.720 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251044220 2024-02-05T08:00:30.720 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510446b0 2024-02-05T08:00:30.720 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251044800 2024-02-05T08:00:30.720 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251044dd0 2024-02-05T08:00:30.720 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251044350 2024-02-05T08:00:30.720 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251044480 2024-02-05T08:00:30.720 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251044930 2024-02-05T08:00:30.720 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251044a60 2024-02-05T08:00:30.720 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251044b90 2024-02-05T08:00:30.720 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251044cc0 2024-02-05T08:00:30.720 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510453a0 2024-02-05T08:00:30.720 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251044f00 2024-02-05T08:00:30.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251045050 2024-02-05T08:00:30.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251045180 2024-02-05T08:00:30.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510454d0 2024-02-05T08:00:30.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251045960 2024-02-05T08:00:30.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251045a90 2024-02-05T08:00:30.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251044590 2024-02-05T08:00:30.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251045f20 2024-02-05T08:00:30.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251045bc0 2024-02-05T08:00:30.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251045cf0 2024-02-05T08:00:30.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251045e00 2024-02-05T08:00:30.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510455e0 2024-02-05T08:00:30.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251045750 2024-02-05T08:00:30.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251043fc0 2024-02-05T08:00:30.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251046030 2024-02-05T08:00:30.721 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510464e0 2024-02-05T08:00:30.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251046610 2024-02-05T08:00:30.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251046be0 2024-02-05T08:00:30.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251046180 2024-02-05T08:00:30.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510462b0 2024-02-05T08:00:30.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251046740 2024-02-05T08:00:30.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251046870 2024-02-05T08:00:30.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510469a0 2024-02-05T08:00:30.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251046ad0 2024-02-05T08:00:30.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510471d0 2024-02-05T08:00:30.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251046d30 2024-02-05T08:00:30.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251046e80 2024-02-05T08:00:30.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251046fb0 2024-02-05T08:00:30.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251047300 2024-02-05T08:00:30.722 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251047790 2024-02-05T08:00:30.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510478c0 2024-02-05T08:00:30.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251047e90 2024-02-05T08:00:30.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251047430 2024-02-05T08:00:30.723 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251047560 2024-02-05T08:00:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:31 smithi157 ceph-mon[33345]: pgmap v675: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-05T08:00:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:31 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:31 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:31 smithi157 ceph-mon[33345]: osdmap e445: 8 total, 8 up, 8 in 2024-02-05T08:00:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:31 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:31 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:31.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:31 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:31 smithi131 ceph-mon[41654]: pgmap v675: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-05T08:00:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:31 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:31 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:31 smithi131 ceph-mon[41654]: osdmap e445: 8 total, 8 up, 8 in 2024-02-05T08:00:31.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:31 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:31.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:31 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:31.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:31 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:31 smithi131 ceph-mon[29507]: pgmap v675: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-05T08:00:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:31 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:31 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:31 smithi131 ceph-mon[29507]: osdmap e445: 8 total, 8 up, 8 in 2024-02-05T08:00:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:31 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:31 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:31 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:00:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:32 smithi157 ceph-mon[33345]: osdmap e446: 8 total, 8 up, 8 in 2024-02-05T08:00:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:32 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:32 smithi157 ceph-mon[33345]: pgmap v678: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-05T08:00:32.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:00:32.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:32 smithi131 ceph-mon[41654]: osdmap e446: 8 total, 8 up, 8 in 2024-02-05T08:00:32.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:32 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:32.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:32.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:32 smithi131 ceph-mon[41654]: pgmap v678: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-05T08:00:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:00:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:32 smithi131 ceph-mon[29507]: osdmap e446: 8 total, 8 up, 8 in 2024-02-05T08:00:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:32 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:32 smithi131 ceph-mon[29507]: pgmap v678: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-05T08:00:33.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:00:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:00:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:00:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T08:00:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:33 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:33 smithi131 ceph-mon[41654]: osdmap e447: 8 total, 8 up, 8 in 2024-02-05T08:00:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T08:00:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T08:00:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:33 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:33 smithi131 ceph-mon[29507]: osdmap e447: 8 total, 8 up, 8 in 2024-02-05T08:00:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:33.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T08:00:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T08:00:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:33 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:33 smithi157 ceph-mon[33345]: osdmap e447: 8 total, 8 up, 8 in 2024-02-05T08:00:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T08:00:34.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:34.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:34 smithi157 ceph-mon[33345]: osdmap e448: 8 total, 8 up, 8 in 2024-02-05T08:00:34.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:34.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:34 smithi157 ceph-mon[33345]: pgmap v681: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:34.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:00:34.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:34 smithi157 ceph-mon[33345]: osdmap e449: 8 total, 8 up, 8 in 2024-02-05T08:00:34.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:34.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:34 smithi131 ceph-mon[41654]: osdmap e448: 8 total, 8 up, 8 in 2024-02-05T08:00:34.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:34.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:34 smithi131 ceph-mon[41654]: pgmap v681: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:34.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:00:34.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:34 smithi131 ceph-mon[41654]: osdmap e449: 8 total, 8 up, 8 in 2024-02-05T08:00:34.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:34.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:34 smithi131 ceph-mon[29507]: osdmap e448: 8 total, 8 up, 8 in 2024-02-05T08:00:34.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:34.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:34 smithi131 ceph-mon[29507]: pgmap v681: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:34.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:00:34.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:34 smithi131 ceph-mon[29507]: osdmap e449: 8 total, 8 up, 8 in 2024-02-05T08:00:35.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-05T08:00:35.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-05T08:00:35.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-05T08:00:35.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-05T08:00:35.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-05T08:00:35.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-05T08:00:36.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-05T08:00:36.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:36 smithi157 ceph-mon[33345]: osdmap e450: 8 total, 8 up, 8 in 2024-02-05T08:00:36.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-05T08:00:36.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-05T08:00:36.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:36 smithi157 ceph-mon[33345]: pgmap v684: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:36.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-05T08:00:36.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:36 smithi131 ceph-mon[41654]: osdmap e450: 8 total, 8 up, 8 in 2024-02-05T08:00:36.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-05T08:00:36.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-05T08:00:36.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:36 smithi131 ceph-mon[41654]: pgmap v684: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:36.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-05T08:00:36.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:36 smithi131 ceph-mon[29507]: osdmap e450: 8 total, 8 up, 8 in 2024-02-05T08:00:36.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-05T08:00:36.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-05T08:00:36.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:36 smithi131 ceph-mon[29507]: pgmap v684: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:37.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-05T08:00:37.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:37 smithi157 ceph-mon[33345]: osdmap e451: 8 total, 8 up, 8 in 2024-02-05T08:00:37.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:37.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:37.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:37 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:37.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-05T08:00:37.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:37 smithi131 ceph-mon[29507]: osdmap e451: 8 total, 8 up, 8 in 2024-02-05T08:00:37.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:37.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:37.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:37 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:37.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-05T08:00:37.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:37 smithi131 ceph-mon[41654]: osdmap e451: 8 total, 8 up, 8 in 2024-02-05T08:00:37.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:37.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:37.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:37 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:38.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:00:38.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:38 smithi157 ceph-mon[33345]: osdmap e452: 8 total, 8 up, 8 in 2024-02-05T08:00:38.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:38 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:38.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:38.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:38 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-81"}]: dispatch 2024-02-05T08:00:38.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-81"}]: dispatch 2024-02-05T08:00:38.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:38 smithi157 ceph-mon[33345]: pgmap v687: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:00:38.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:00:38.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:38 smithi131 ceph-mon[29507]: osdmap e452: 8 total, 8 up, 8 in 2024-02-05T08:00:38.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:38 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:38.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:38.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:38 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-81"}]: dispatch 2024-02-05T08:00:38.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-81"}]: dispatch 2024-02-05T08:00:38.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:38 smithi131 ceph-mon[29507]: pgmap v687: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:00:38.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:00:38.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:38 smithi131 ceph-mon[41654]: osdmap e452: 8 total, 8 up, 8 in 2024-02-05T08:00:38.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:38 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:38.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:38.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:38 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-81"}]: dispatch 2024-02-05T08:00:38.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-81"}]: dispatch 2024-02-05T08:00:38.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:38 smithi131 ceph-mon[41654]: pgmap v687: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:00:39.111 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:00:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:00:39.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:39 smithi157 ceph-mon[33345]: osdmap e453: 8 total, 8 up, 8 in 2024-02-05T08:00:39.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:39 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:39.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:39 smithi131 ceph-mon[41654]: osdmap e453: 8 total, 8 up, 8 in 2024-02-05T08:00:39.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:39 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:39.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:39 smithi131 ceph-mon[29507]: osdmap e453: 8 total, 8 up, 8 in 2024-02-05T08:00:39.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:39 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:40.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:40 smithi157 ceph-mon[33345]: osdmap e454: 8 total, 8 up, 8 in 2024-02-05T08:00:40.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:40 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:40.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:40.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:40 smithi157 ceph-mon[33345]: pgmap v690: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-05T08:00:40.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:40 smithi131 ceph-mon[41654]: osdmap e454: 8 total, 8 up, 8 in 2024-02-05T08:00:40.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:40 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:40.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:40.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:40 smithi131 ceph-mon[41654]: pgmap v690: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-05T08:00:40.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:40 smithi131 ceph-mon[29507]: osdmap e454: 8 total, 8 up, 8 in 2024-02-05T08:00:40.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:40 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:40.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:40.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:40 smithi131 ceph-mon[29507]: pgmap v690: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-05T08:00:41.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:41.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:41 smithi157 ceph-mon[33345]: osdmap e455: 8 total, 8 up, 8 in 2024-02-05T08:00:41.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:41.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:41.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:41.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:41.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:41 smithi131 ceph-mon[41654]: osdmap e455: 8 total, 8 up, 8 in 2024-02-05T08:00:41.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:41.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:41.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:41.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:41.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:41 smithi131 ceph-mon[29507]: osdmap e455: 8 total, 8 up, 8 in 2024-02-05T08:00:41.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:41.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:41.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:42.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:00:42.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:42 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:42.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:42 smithi157 ceph-mon[33345]: osdmap e456: 8 total, 8 up, 8 in 2024-02-05T08:00:42.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:42.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:42 smithi157 ceph-mon[33345]: pgmap v693: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-05T08:00:42.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:42 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:42.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:00:42.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:42 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:42.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:42 smithi131 ceph-mon[41654]: osdmap e456: 8 total, 8 up, 8 in 2024-02-05T08:00:42.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:42.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:42 smithi131 ceph-mon[41654]: pgmap v693: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-05T08:00:42.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:42 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:42.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:00:42.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:42 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:42.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:42 smithi131 ceph-mon[29507]: osdmap e456: 8 total, 8 up, 8 in 2024-02-05T08:00:42.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:42.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:42 smithi131 ceph-mon[29507]: pgmap v693: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-05T08:00:42.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:42 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:43.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:00:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:00:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:00:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T08:00:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:43 smithi131 ceph-mon[41654]: osdmap e457: 8 total, 8 up, 8 in 2024-02-05T08:00:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T08:00:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:43 smithi131 ceph-mon[29507]: osdmap e457: 8 total, 8 up, 8 in 2024-02-05T08:00:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:43.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T08:00:43.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:43 smithi157 ceph-mon[33345]: osdmap e457: 8 total, 8 up, 8 in 2024-02-05T08:00:43.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:43.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:44.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T08:00:44.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:44.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:44 smithi157 ceph-mon[33345]: osdmap e458: 8 total, 8 up, 8 in 2024-02-05T08:00:44.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:44.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:44 smithi157 ceph-mon[33345]: pgmap v696: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:44.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T08:00:44.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:44.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:44 smithi131 ceph-mon[41654]: osdmap e458: 8 total, 8 up, 8 in 2024-02-05T08:00:44.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:44.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:44 smithi131 ceph-mon[41654]: pgmap v696: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:44.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T08:00:44.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:44.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:44 smithi131 ceph-mon[29507]: osdmap e458: 8 total, 8 up, 8 in 2024-02-05T08:00:44.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:44.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:44 smithi131 ceph-mon[29507]: pgmap v696: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:00:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:45 smithi157 ceph-mon[33345]: osdmap e459: 8 total, 8 up, 8 in 2024-02-05T08:00:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:45 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:45.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:00:45.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:45 smithi131 ceph-mon[41654]: osdmap e459: 8 total, 8 up, 8 in 2024-02-05T08:00:45.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:45 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:00:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:45 smithi131 ceph-mon[29507]: osdmap e459: 8 total, 8 up, 8 in 2024-02-05T08:00:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:45 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:46.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:46 smithi157 ceph-mon[33345]: osdmap e460: 8 total, 8 up, 8 in 2024-02-05T08:00:46.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:46 smithi157 ceph-mon[33345]: pgmap v699: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:46.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:46 smithi131 ceph-mon[41654]: osdmap e460: 8 total, 8 up, 8 in 2024-02-05T08:00:46.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:46 smithi131 ceph-mon[41654]: pgmap v699: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:46.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:46 smithi131 ceph-mon[29507]: osdmap e460: 8 total, 8 up, 8 in 2024-02-05T08:00:46.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:46 smithi131 ceph-mon[29507]: pgmap v699: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:47.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:47 smithi157 ceph-mon[33345]: osdmap e461: 8 total, 8 up, 8 in 2024-02-05T08:00:47.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:47 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:47.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:47.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:47 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-83"}]: dispatch 2024-02-05T08:00:47.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-83"}]: dispatch 2024-02-05T08:00:47.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:47 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:47.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:47 smithi131 ceph-mon[29507]: osdmap e461: 8 total, 8 up, 8 in 2024-02-05T08:00:47.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:47 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:47.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:47.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:47 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-83"}]: dispatch 2024-02-05T08:00:47.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-83"}]: dispatch 2024-02-05T08:00:47.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:47 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:47 smithi131 ceph-mon[41654]: osdmap e461: 8 total, 8 up, 8 in 2024-02-05T08:00:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:47 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:47 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-83"}]: dispatch 2024-02-05T08:00:47.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-83"}]: dispatch 2024-02-05T08:00:47.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:47 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:48.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:48 smithi157 ceph-mon[33345]: osdmap e462: 8 total, 8 up, 8 in 2024-02-05T08:00:48.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:48 smithi157 ceph-mon[33345]: pgmap v702: 260 pgs: 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:00:48.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:48 smithi157 ceph-mon[33345]: osdmap e463: 8 total, 8 up, 8 in 2024-02-05T08:00:48.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:48.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:48.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:48 smithi131 ceph-mon[41654]: osdmap e462: 8 total, 8 up, 8 in 2024-02-05T08:00:48.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:48 smithi131 ceph-mon[41654]: pgmap v702: 260 pgs: 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:00:48.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:48 smithi131 ceph-mon[41654]: osdmap e463: 8 total, 8 up, 8 in 2024-02-05T08:00:48.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:48.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:48.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:48 smithi131 ceph-mon[29507]: osdmap e462: 8 total, 8 up, 8 in 2024-02-05T08:00:48.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:48 smithi131 ceph-mon[29507]: pgmap v702: 260 pgs: 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:00:48.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:48 smithi131 ceph-mon[29507]: osdmap e463: 8 total, 8 up, 8 in 2024-02-05T08:00:48.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:48.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:49.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:00:48 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:00:49.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:49 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:49.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:49.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:49 smithi157 ceph-mon[33345]: osdmap e464: 8 total, 8 up, 8 in 2024-02-05T08:00:49.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:49.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:49.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:49.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:49 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:49.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:49.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:49 smithi131 ceph-mon[41654]: osdmap e464: 8 total, 8 up, 8 in 2024-02-05T08:00:49.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:49.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:49.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:49.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:49 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:49.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:49.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:49 smithi131 ceph-mon[29507]: osdmap e464: 8 total, 8 up, 8 in 2024-02-05T08:00:49.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:49.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:49.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:00:50.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:50 smithi157 ceph-mon[33345]: pgmap v705: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-02-05T08:00:50.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:00:50.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:50.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:50 smithi157 ceph-mon[33345]: osdmap e465: 8 total, 8 up, 8 in 2024-02-05T08:00:50.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:50.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:50 smithi131 ceph-mon[41654]: pgmap v705: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-02-05T08:00:50.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:00:50.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:50.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:50 smithi131 ceph-mon[41654]: osdmap e465: 8 total, 8 up, 8 in 2024-02-05T08:00:50.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:50.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:50 smithi131 ceph-mon[29507]: pgmap v705: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-02-05T08:00:50.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:00:50.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:50.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:50 smithi131 ceph-mon[29507]: osdmap e465: 8 total, 8 up, 8 in 2024-02-05T08:00:50.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:52.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T08:00:52.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:52.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:52 smithi157 ceph-mon[33345]: osdmap e466: 8 total, 8 up, 8 in 2024-02-05T08:00:52.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:52.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:52 smithi157 ceph-mon[33345]: pgmap v708: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-02-05T08:00:52.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T08:00:52.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:52.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:52 smithi131 ceph-mon[29507]: osdmap e466: 8 total, 8 up, 8 in 2024-02-05T08:00:52.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:52.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:52 smithi131 ceph-mon[29507]: pgmap v708: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-02-05T08:00:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_tier","val": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T08:00:52.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:52.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:52 smithi131 ceph-mon[41654]: osdmap e466: 8 total, 8 up, 8 in 2024-02-05T08:00:52.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:00:52.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:52 smithi131 ceph-mon[41654]: pgmap v708: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-02-05T08:00:53.203 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:00:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:00:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:00:53.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T08:00:53.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:53.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:53 smithi157 ceph-mon[33345]: osdmap e467: 8 total, 8 up, 8 in 2024-02-05T08:00:53.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:53.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T08:00:53.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:53.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:53 smithi131 ceph-mon[29507]: osdmap e467: 8 total, 8 up, 8 in 2024-02-05T08:00:53.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:53.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T08:00:53.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:53.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:53 smithi131 ceph-mon[41654]: osdmap e467: 8 total, 8 up, 8 in 2024-02-05T08:00:53.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:00:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:00:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:54 smithi157 ceph-mon[33345]: osdmap e468: 8 total, 8 up, 8 in 2024-02-05T08:00:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:54 smithi157 ceph-mon[33345]: pgmap v711: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T08:00:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:00:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:54 smithi131 ceph-mon[41654]: osdmap e468: 8 total, 8 up, 8 in 2024-02-05T08:00:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:54 smithi131 ceph-mon[41654]: pgmap v711: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T08:00:54.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:00:54.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:54 smithi131 ceph-mon[29507]: osdmap e468: 8 total, 8 up, 8 in 2024-02-05T08:00:54.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:54 smithi131 ceph-mon[29507]: pgmap v711: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T08:00:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:55 smithi157 ceph-mon[33345]: osdmap e469: 8 total, 8 up, 8 in 2024-02-05T08:00:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:55 smithi131 ceph-mon[41654]: osdmap e469: 8 total, 8 up, 8 in 2024-02-05T08:00:55.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:55 smithi131 ceph-mon[29507]: osdmap e469: 8 total, 8 up, 8 in 2024-02-05T08:00:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:56 smithi157 ceph-mon[33345]: osdmap e470: 8 total, 8 up, 8 in 2024-02-05T08:00:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-85"}]: dispatch 2024-02-05T08:00:56.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-85"}]: dispatch 2024-02-05T08:00:56.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:56 smithi157 ceph-mon[33345]: pgmap v714: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T08:00:56.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:56 smithi131 ceph-mon[41654]: osdmap e470: 8 total, 8 up, 8 in 2024-02-05T08:00:56.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:56.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:56.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-85"}]: dispatch 2024-02-05T08:00:56.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-85"}]: dispatch 2024-02-05T08:00:56.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:56 smithi131 ceph-mon[41654]: pgmap v714: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T08:00:56.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:56 smithi131 ceph-mon[29507]: osdmap e470: 8 total, 8 up, 8 in 2024-02-05T08:00:56.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:56.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:00:56.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-85"}]: dispatch 2024-02-05T08:00:56.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-85"}]: dispatch 2024-02-05T08:00:56.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:56 smithi131 ceph-mon[29507]: pgmap v714: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T08:00:57.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:57 smithi131 ceph-mon[41654]: osdmap e471: 8 total, 8 up, 8 in 2024-02-05T08:00:57.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:57 smithi131 ceph-mon[29507]: osdmap e471: 8 total, 8 up, 8 in 2024-02-05T08:00:57.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:57 smithi157 ceph-mon[33345]: osdmap e471: 8 total, 8 up, 8 in 2024-02-05T08:00:58.249 INFO:tasks.workunit.client.0.smithi131.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18090 ms) 2024-02-05T08:00:58.249 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-02-05T08:00:58.250 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22729 ms) 2024-02-05T08:00:58.250 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-02-05T08:00:58.250 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3018 ms) 2024-02-05T08:00:58.250 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-02-05T08:00:58.250 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:00:58.250 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3006 ms) 2024-02-05T08:00:58.250 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-02-05T08:00:58.250 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3008 ms) 2024-02-05T08:00:58.250 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-02-05T08:00:58.250 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:00:58.250 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3025 ms) 2024-02-05T08:00:58.250 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-02-05T08:00:58.250 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:00:58.250 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4026 ms) 2024-02-05T08:00:58.250 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-02-05T08:00:58.251 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:00:58.251 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (39920 ms) 2024-02-05T08:00:58.251 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-02-05T08:00:58.251 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:00:58.251 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (17062 ms) 2024-02-05T08:00:58.251 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-02-05T08:00:58.251 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:00:58.251 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4043 ms) 2024-02-05T08:00:58.251 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-02-05T08:00:58.251 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:00:58.251 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3007 ms) 2024-02-05T08:00:58.251 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-02-05T08:00:58.251 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:00:58.251 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (25079 ms) 2024-02-05T08:00:58.251 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-02-05T08:00:58.251 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:00:58.252 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15078 ms) 2024-02-05T08:00:58.252 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-02-05T08:00:58.252 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:00:58.252 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5028 ms) 2024-02-05T08:00:58.252 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-02-05T08:00:58.252 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:00:58.252 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4020 ms) 2024-02-05T08:00:58.252 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-02-05T08:00:58.252 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:00:58.252 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: waiting for scrubs... 2024-02-05T08:00:58.252 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: done waiting 2024-02-05T08:00:58.252 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25096 ms) 2024-02-05T08:00:58.252 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-02-05T08:00:58.253 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:00:58.253 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10076 ms) 2024-02-05T08:00:58.253 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-02-05T08:00:58.253 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:00:58.253 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9069 ms) 2024-02-05T08:00:58.253 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-02-05T08:00:58.253 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:00:58.253 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9044 ms) 2024-02-05T08:00:58.253 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-02-05T08:00:58.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:58 smithi131 ceph-mon[41654]: osdmap e472: 8 total, 8 up, 8 in 2024-02-05T08:00:58.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:58.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:58.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:58 smithi131 ceph-mon[41654]: pgmap v717: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:00:58.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:58 smithi131 ceph-mon[29507]: osdmap e472: 8 total, 8 up, 8 in 2024-02-05T08:00:58.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:58.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:58.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:58 smithi131 ceph-mon[29507]: pgmap v717: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:00:58.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:58 smithi157 ceph-mon[33345]: osdmap e472: 8 total, 8 up, 8 in 2024-02-05T08:00:58.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:58.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:00:58.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:58 smithi157 ceph-mon[33345]: pgmap v717: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:00:59.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:00:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:00:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:59 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:59 smithi131 ceph-mon[29507]: osdmap e473: 8 total, 8 up, 8 in 2024-02-05T08:00:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:00:59 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:59 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:59.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:59 smithi131 ceph-mon[41654]: osdmap e473: 8 total, 8 up, 8 in 2024-02-05T08:00:59.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:59.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:00:59 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:59.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:59 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:00:59.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:00:59.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:59 smithi157 ceph-mon[33345]: osdmap e473: 8 total, 8 up, 8 in 2024-02-05T08:00:59.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:00:59.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:00:59 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:00 smithi131 ceph-mon[41654]: osdmap e474: 8 total, 8 up, 8 in 2024-02-05T08:01:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:01:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:01:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:00 smithi131 ceph-mon[41654]: pgmap v720: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-02-05T08:01:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:00.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:00 smithi131 ceph-mon[29507]: osdmap e474: 8 total, 8 up, 8 in 2024-02-05T08:01:00.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:01:00.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:01:00.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:00 smithi131 ceph-mon[29507]: pgmap v720: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-02-05T08:01:00.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:00.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:00 smithi157 ceph-mon[33345]: osdmap e474: 8 total, 8 up, 8 in 2024-02-05T08:01:00.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:01:00.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:01:00.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:00 smithi157 ceph-mon[33345]: pgmap v720: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-02-05T08:01:00.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:01:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_tier","val": "test-rados-api-smithi131-82290-89-test-flush"}]: dispatch 2024-02-05T08:01:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:01 smithi131 ceph-mon[41654]: osdmap e475: 8 total, 8 up, 8 in 2024-02-05T08:01:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_tier","val": "test-rados-api-smithi131-82290-89-test-flush"}]: dispatch 2024-02-05T08:01:01.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:01:01.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_tier","val": "test-rados-api-smithi131-82290-89-test-flush"}]: dispatch 2024-02-05T08:01:01.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:01 smithi131 ceph-mon[29507]: osdmap e475: 8 total, 8 up, 8 in 2024-02-05T08:01:01.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_tier","val": "test-rados-api-smithi131-82290-89-test-flush"}]: dispatch 2024-02-05T08:01:01.712 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:01:01.712 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_tier","val": "test-rados-api-smithi131-82290-89-test-flush"}]: dispatch 2024-02-05T08:01:01.712 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:01 smithi157 ceph-mon[33345]: osdmap e475: 8 total, 8 up, 8 in 2024-02-05T08:01:01.712 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_tier","val": "test-rados-api-smithi131-82290-89-test-flush"}]: dispatch 2024-02-05T08:01:02.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_tier","val": "test-rados-api-smithi131-82290-89-test-flush"}]': finished 2024-02-05T08:01:02.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:01:02.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:02 smithi157 ceph-mon[33345]: osdmap e476: 8 total, 8 up, 8 in 2024-02-05T08:01:02.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:01:02.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:02 smithi157 ceph-mon[33345]: pgmap v723: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-05T08:01:02.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T08:01:02.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:01:02.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:02 smithi157 ceph-mon[33345]: osdmap e477: 8 total, 8 up, 8 in 2024-02-05T08:01:02.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_tier","val": "test-rados-api-smithi131-82290-89-test-flush"}]': finished 2024-02-05T08:01:02.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:01:02.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-mon[41654]: osdmap e476: 8 total, 8 up, 8 in 2024-02-05T08:01:02.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:01:02.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-mon[41654]: pgmap v723: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-05T08:01:02.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T08:01:02.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:01:02.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-mon[41654]: osdmap e477: 8 total, 8 up, 8 in 2024-02-05T08:01:02.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_tier","val": "test-rados-api-smithi131-82290-89-test-flush"}]': finished 2024-02-05T08:01:02.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:01:02.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-mon[29507]: osdmap e476: 8 total, 8 up, 8 in 2024-02-05T08:01:02.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:01:02.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-mon[29507]: pgmap v723: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-05T08:01:02.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T08:01:02.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:01:02.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-mon[29507]: osdmap e477: 8 total, 8 up, 8 in 2024-02-05T08:01:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:01:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:01:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:01:03.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:01:03.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:01:03.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:03 smithi157 ceph-mon[33345]: osdmap e478: 8 total, 8 up, 8 in 2024-02-05T08:01:03.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:01:03.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:01:03.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:03 smithi131 ceph-mon[41654]: osdmap e478: 8 total, 8 up, 8 in 2024-02-05T08:01:03.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:01:03.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:01:03.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:03 smithi131 ceph-mon[29507]: osdmap e478: 8 total, 8 up, 8 in 2024-02-05T08:01:04.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:04 smithi157 ceph-mon[33345]: pgmap v726: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T08:01:04.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:04 smithi131 ceph-mon[41654]: pgmap v726: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T08:01:04.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:04 smithi131 ceph-mon[29507]: pgmap v726: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T08:01:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:05 smithi157 ceph-mon[33345]: osdmap e479: 8 total, 8 up, 8 in 2024-02-05T08:01:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-87"}]: dispatch 2024-02-05T08:01:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-87"}]: dispatch 2024-02-05T08:01:05.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:05 smithi131 ceph-mon[41654]: osdmap e479: 8 total, 8 up, 8 in 2024-02-05T08:01:05.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:05.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:05.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-87"}]: dispatch 2024-02-05T08:01:05.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-87"}]: dispatch 2024-02-05T08:01:05.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:05 smithi131 ceph-mon[29507]: osdmap e479: 8 total, 8 up, 8 in 2024-02-05T08:01:05.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:05.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:05.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-87"}]: dispatch 2024-02-05T08:01:05.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-87"}]: dispatch 2024-02-05T08:01:06.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:06 smithi157 ceph-mon[33345]: osdmap e480: 8 total, 8 up, 8 in 2024-02-05T08:01:06.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:06 smithi157 ceph-mon[33345]: pgmap v729: 260 pgs: 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:01:06.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:06 smithi131 ceph-mon[41654]: osdmap e480: 8 total, 8 up, 8 in 2024-02-05T08:01:06.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:06 smithi131 ceph-mon[41654]: pgmap v729: 260 pgs: 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:01:06.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:06 smithi131 ceph-mon[29507]: osdmap e480: 8 total, 8 up, 8 in 2024-02-05T08:01:06.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:06 smithi131 ceph-mon[29507]: pgmap v729: 260 pgs: 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:01:07.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:07 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:07.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:07 smithi157 ceph-mon[33345]: osdmap e481: 8 total, 8 up, 8 in 2024-02-05T08:01:07.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:07 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:07.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:07.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:07 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:07.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:07 smithi131 ceph-mon[41654]: osdmap e481: 8 total, 8 up, 8 in 2024-02-05T08:01:07.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:07 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:07.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:07.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:07 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:07.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:07 smithi131 ceph-mon[29507]: osdmap e481: 8 total, 8 up, 8 in 2024-02-05T08:01:07.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:07 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:07.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:01:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:01:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:08 smithi157 ceph-mon[33345]: osdmap e482: 8 total, 8 up, 8 in 2024-02-05T08:01:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:01:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:01:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:08 smithi157 ceph-mon[33345]: pgmap v732: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:01:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:01:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:01:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:08 smithi131 ceph-mon[41654]: osdmap e482: 8 total, 8 up, 8 in 2024-02-05T08:01:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:01:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:01:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:08 smithi131 ceph-mon[41654]: pgmap v732: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:01:08.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:01:08.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:01:08.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:08 smithi131 ceph-mon[29507]: osdmap e482: 8 total, 8 up, 8 in 2024-02-05T08:01:08.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:01:08.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:01:08.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:08 smithi131 ceph-mon[29507]: pgmap v732: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:01:09.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:01:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:01:09.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:01:09.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:09 smithi157 ceph-mon[33345]: osdmap e483: 8 total, 8 up, 8 in 2024-02-05T08:01:09.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:09 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:01:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:09 smithi131 ceph-mon[29507]: osdmap e483: 8 total, 8 up, 8 in 2024-02-05T08:01:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:09 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:09.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:01:09.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:09 smithi131 ceph-mon[41654]: osdmap e483: 8 total, 8 up, 8 in 2024-02-05T08:01:09.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:09 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:10.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:10 smithi157 ceph-mon[33345]: osdmap e484: 8 total, 8 up, 8 in 2024-02-05T08:01:10.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:10 smithi157 ceph-mon[33345]: pgmap v735: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-05T08:01:10.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:10 smithi131 ceph-mon[41654]: osdmap e484: 8 total, 8 up, 8 in 2024-02-05T08:01:10.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:10 smithi131 ceph-mon[41654]: pgmap v735: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-05T08:01:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:10 smithi131 ceph-mon[29507]: osdmap e484: 8 total, 8 up, 8 in 2024-02-05T08:01:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:10 smithi131 ceph-mon[29507]: pgmap v735: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-05T08:01:11.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:11 smithi157 ceph-mon[33345]: osdmap e485: 8 total, 8 up, 8 in 2024-02-05T08:01:11.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:11.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:11.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-90"}]: dispatch 2024-02-05T08:01:11.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-90"}]: dispatch 2024-02-05T08:01:11.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:11 smithi131 ceph-mon[41654]: osdmap e485: 8 total, 8 up, 8 in 2024-02-05T08:01:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-90"}]: dispatch 2024-02-05T08:01:11.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-90"}]: dispatch 2024-02-05T08:01:11.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:11 smithi131 ceph-mon[29507]: osdmap e485: 8 total, 8 up, 8 in 2024-02-05T08:01:11.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:11.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:11.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-90"}]: dispatch 2024-02-05T08:01:11.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-90"}]: dispatch 2024-02-05T08:01:12.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:12 smithi157 ceph-mon[33345]: osdmap e486: 8 total, 8 up, 8 in 2024-02-05T08:01:12.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:12 smithi157 ceph-mon[33345]: pgmap v738: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:01:12.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:12 smithi131 ceph-mon[41654]: osdmap e486: 8 total, 8 up, 8 in 2024-02-05T08:01:12.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:12 smithi131 ceph-mon[41654]: pgmap v738: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:01:12.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:12 smithi131 ceph-mon[29507]: osdmap e486: 8 total, 8 up, 8 in 2024-02-05T08:01:12.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:12 smithi131 ceph-mon[29507]: pgmap v738: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:01:13.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:01:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:01:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:01:13.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:13 smithi157 ceph-mon[33345]: osdmap e487: 8 total, 8 up, 8 in 2024-02-05T08:01:13.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:13 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:13.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:13 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:13.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:13 smithi131 ceph-mon[41654]: osdmap e487: 8 total, 8 up, 8 in 2024-02-05T08:01:13.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:13 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:13.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:13 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:13.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:13 smithi131 ceph-mon[29507]: osdmap e487: 8 total, 8 up, 8 in 2024-02-05T08:01:13.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:13 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:13.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:13 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:14.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:01:14.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:14 smithi157 ceph-mon[33345]: osdmap e488: 8 total, 8 up, 8 in 2024-02-05T08:01:14.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:01:14.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:14 smithi157 ceph-mon[33345]: pgmap v741: 292 pgs: 3 creating+activating, 29 creating+peering, 260 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:01:14.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:01:14.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:14 smithi131 ceph-mon[29507]: osdmap e488: 8 total, 8 up, 8 in 2024-02-05T08:01:14.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:01:14.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:14 smithi131 ceph-mon[29507]: pgmap v741: 292 pgs: 3 creating+activating, 29 creating+peering, 260 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:01:14.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:01:14.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:14 smithi131 ceph-mon[41654]: osdmap e488: 8 total, 8 up, 8 in 2024-02-05T08:01:14.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:01:14.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:14 smithi131 ceph-mon[41654]: pgmap v741: 292 pgs: 3 creating+activating, 29 creating+peering, 260 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:01:15.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:15 smithi157 ceph-mon[33345]: osdmap e489: 8 total, 8 up, 8 in 2024-02-05T08:01:15.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:15.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:01:15.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:15 smithi131 ceph-mon[29507]: osdmap e489: 8 total, 8 up, 8 in 2024-02-05T08:01:15.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:15.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:01:15.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:15 smithi131 ceph-mon[41654]: osdmap e489: 8 total, 8 up, 8 in 2024-02-05T08:01:15.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:15.769 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:01:16.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: osdmap e490: 8 total, 8 up, 8 in 2024-02-05T08:01:16.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: Adjusting osd_memory_target on smithi131 to 3248M 2024-02-05T08:01:16.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:01:16.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.722 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: Adjusting osd_memory_target on smithi157 to 3248M 2024-02-05T08:01:16.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:01:16.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:01:16.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:01:16.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:16.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:16.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:01:16.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-92"}]: dispatch 2024-02-05T08:01:16.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-92"}]: dispatch 2024-02-05T08:01:16.723 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:16 smithi157 ceph-mon[33345]: pgmap v744: 292 pgs: 3 creating+activating, 29 creating+peering, 260 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:01:16.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: osdmap e490: 8 total, 8 up, 8 in 2024-02-05T08:01:16.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: Adjusting osd_memory_target on smithi131 to 3248M 2024-02-05T08:01:16.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:01:16.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: Adjusting osd_memory_target on smithi157 to 3248M 2024-02-05T08:01:16.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:01:16.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:01:16.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:01:16.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:16.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:16.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:01:16.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: osdmap e490: 8 total, 8 up, 8 in 2024-02-05T08:01:16.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: Adjusting osd_memory_target on smithi131 to 3248M 2024-02-05T08:01:16.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:01:16.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T08:01:16.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: Adjusting osd_memory_target on smithi157 to 3248M 2024-02-05T08:01:16.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:01:16.772 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:01:16.772 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:01:16.772 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:16.772 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:16.772 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:01:16.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-92"}]: dispatch 2024-02-05T08:01:16.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-92"}]: dispatch 2024-02-05T08:01:16.772 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[29507]: pgmap v744: 292 pgs: 3 creating+activating, 29 creating+peering, 260 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:01:16.773 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-92"}]: dispatch 2024-02-05T08:01:16.773 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-92"}]: dispatch 2024-02-05T08:01:16.773 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:16 smithi131 ceph-mon[41654]: pgmap v744: 292 pgs: 3 creating+activating, 29 creating+peering, 260 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:01:17.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:17 smithi157 ceph-mon[33345]: osdmap e491: 8 total, 8 up, 8 in 2024-02-05T08:01:17.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:17 smithi131 ceph-mon[41654]: osdmap e491: 8 total, 8 up, 8 in 2024-02-05T08:01:17.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:17 smithi131 ceph-mon[29507]: osdmap e491: 8 total, 8 up, 8 in 2024-02-05T08:01:18.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:18 smithi157 ceph-mon[33345]: osdmap e492: 8 total, 8 up, 8 in 2024-02-05T08:01:18.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:18.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:18.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:18 smithi157 ceph-mon[33345]: pgmap v747: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:01:18.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:01:18.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:18 smithi131 ceph-mon[41654]: osdmap e492: 8 total, 8 up, 8 in 2024-02-05T08:01:18.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:18.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:18.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:18 smithi131 ceph-mon[41654]: pgmap v747: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:01:18.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:01:18.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:18 smithi131 ceph-mon[29507]: osdmap e492: 8 total, 8 up, 8 in 2024-02-05T08:01:18.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:18.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:18.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:18 smithi131 ceph-mon[29507]: pgmap v747: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:01:18.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:01:19.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:01:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:01:19.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:19 smithi157 ceph-mon[33345]: osdmap e493: 8 total, 8 up, 8 in 2024-02-05T08:01:19.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:01:19.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:19 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:19.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:19 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:19.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:19 smithi157 ceph-mon[33345]: osdmap e494: 8 total, 8 up, 8 in 2024-02-05T08:01:19.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:19 smithi131 ceph-mon[41654]: osdmap e493: 8 total, 8 up, 8 in 2024-02-05T08:01:19.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:01:19.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:19 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:19.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:19 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:19.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:19 smithi131 ceph-mon[41654]: osdmap e494: 8 total, 8 up, 8 in 2024-02-05T08:01:19.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:19 smithi131 ceph-mon[29507]: osdmap e493: 8 total, 8 up, 8 in 2024-02-05T08:01:19.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:01:19.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:19 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:19.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:19 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:19.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:19 smithi131 ceph-mon[29507]: osdmap e494: 8 total, 8 up, 8 in 2024-02-05T08:01:20.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:20 smithi157 ceph-mon[33345]: pgmap v750: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T08:01:20.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:20 smithi131 ceph-mon[41654]: pgmap v750: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T08:01:20.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:20 smithi131 ceph-mon[29507]: pgmap v750: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T08:01:21.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:21 smithi157 ceph-mon[33345]: osdmap e495: 8 total, 8 up, 8 in 2024-02-05T08:01:21.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:21 smithi131 ceph-mon[29507]: osdmap e495: 8 total, 8 up, 8 in 2024-02-05T08:01:21.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:21 smithi131 ceph-mon[41654]: osdmap e495: 8 total, 8 up, 8 in 2024-02-05T08:01:22.943 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:22 smithi131 ceph-mon[41654]: pgmap v752: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 755 B/s wr, 2 op/s 2024-02-05T08:01:22.943 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:22 smithi131 ceph-mon[29507]: pgmap v752: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 755 B/s wr, 2 op/s 2024-02-05T08:01:22.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:22 smithi157 ceph-mon[33345]: pgmap v752: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 755 B/s wr, 2 op/s 2024-02-05T08:01:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:01:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:01:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:01:24.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:24 smithi157 ceph-mon[33345]: pgmap v753: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-02-05T08:01:25.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:24 smithi131 ceph-mon[41654]: pgmap v753: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-02-05T08:01:25.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:24 smithi131 ceph-mon[29507]: pgmap v753: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-02-05T08:01:26.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:26 smithi157 ceph-mon[33345]: pgmap v754: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-05T08:01:26.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:26 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:27.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:26 smithi131 ceph-mon[41654]: pgmap v754: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-05T08:01:27.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:26 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:27.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:26 smithi131 ceph-mon[29507]: pgmap v754: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-05T08:01:27.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:26 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:28.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:28 smithi157 ceph-mon[33345]: pgmap v755: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1017 B/s wr, 3 op/s 2024-02-05T08:01:28.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:01:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:01:29.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:28 smithi131 ceph-mon[29507]: pgmap v755: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1017 B/s wr, 3 op/s 2024-02-05T08:01:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:28 smithi131 ceph-mon[41654]: pgmap v755: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1017 B/s wr, 3 op/s 2024-02-05T08:01:29.873 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:29 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:29.873 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 08:01:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=cleanup t=2024-02-05T08:01:29.49395483Z level=info msg="Completed cleanup jobs" duration=2.410641ms 2024-02-05T08:01:30.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:29 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:30.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:29 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:30.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:30 smithi157 ceph-mon[33345]: pgmap v756: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-02-05T08:01:30.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:30 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:31.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:30 smithi131 ceph-mon[41654]: pgmap v756: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-02-05T08:01:31.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:30 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:31.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:30 smithi131 ceph-mon[29507]: pgmap v756: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-02-05T08:01:31.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:30 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:31.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:31 smithi157 ceph-mon[33345]: osdmap e496: 8 total, 8 up, 8 in 2024-02-05T08:01:32.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:31 smithi131 ceph-mon[41654]: osdmap e496: 8 total, 8 up, 8 in 2024-02-05T08:01:32.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:31 smithi131 ceph-mon[29507]: osdmap e496: 8 total, 8 up, 8 in 2024-02-05T08:01:32.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:32 smithi157 ceph-mon[33345]: pgmap v758: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-02-05T08:01:33.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:32 smithi131 ceph-mon[41654]: pgmap v758: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-02-05T08:01:33.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:32 smithi131 ceph-mon[29507]: pgmap v758: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-02-05T08:01:33.018 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:01:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:01:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:01:34.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:34 smithi157 ceph-mon[33345]: pgmap v759: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:01:35.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:34 smithi131 ceph-mon[41654]: pgmap v759: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:01:35.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:34 smithi131 ceph-mon[29507]: pgmap v759: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:01:36.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:36 smithi157 ceph-mon[33345]: pgmap v760: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:01:36.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:36 smithi157 ceph-mon[33345]: osdmap e497: 8 total, 8 up, 8 in 2024-02-05T08:01:37.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:36 smithi131 ceph-mon[41654]: pgmap v760: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:01:37.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:36 smithi131 ceph-mon[41654]: osdmap e497: 8 total, 8 up, 8 in 2024-02-05T08:01:37.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:36 smithi131 ceph-mon[29507]: pgmap v760: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:01:37.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:36 smithi131 ceph-mon[29507]: osdmap e497: 8 total, 8 up, 8 in 2024-02-05T08:01:38.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:38 smithi157 ceph-mon[33345]: pgmap v762: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:01:38.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:01:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:01:39.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:38 smithi131 ceph-mon[41654]: pgmap v762: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:01:39.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:38 smithi131 ceph-mon[29507]: pgmap v762: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:01:40.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:39 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:40.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:39 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:40.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:39 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:41.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:40 smithi131 ceph-mon[41654]: pgmap v763: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:01:41.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:40 smithi131 ceph-mon[29507]: pgmap v763: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:01:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:40 smithi157 ceph-mon[33345]: pgmap v763: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:01:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-94"}]: dispatch 2024-02-05T08:01:41.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-94"}]: dispatch 2024-02-05T08:01:42.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:42.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:42.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-94"}]: dispatch 2024-02-05T08:01:42.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-94"}]: dispatch 2024-02-05T08:01:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-94"}]: dispatch 2024-02-05T08:01:42.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-94"}]: dispatch 2024-02-05T08:01:43.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:42 smithi131 ceph-mon[41654]: pgmap v764: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:01:43.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:42 smithi131 ceph-mon[41654]: osdmap e498: 8 total, 8 up, 8 in 2024-02-05T08:01:43.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:42 smithi131 ceph-mon[29507]: pgmap v764: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:01:43.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:42 smithi131 ceph-mon[29507]: osdmap e498: 8 total, 8 up, 8 in 2024-02-05T08:01:43.019 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:01:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:01:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:01:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:42 smithi157 ceph-mon[33345]: pgmap v764: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:01:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:42 smithi157 ceph-mon[33345]: osdmap e498: 8 total, 8 up, 8 in 2024-02-05T08:01:44.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:43 smithi131 ceph-mon[41654]: osdmap e499: 8 total, 8 up, 8 in 2024-02-05T08:01:44.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:44.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:44.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:43 smithi131 ceph-mon[29507]: osdmap e499: 8 total, 8 up, 8 in 2024-02-05T08:01:44.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:44.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:44.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:43 smithi157 ceph-mon[33345]: osdmap e499: 8 total, 8 up, 8 in 2024-02-05T08:01:44.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:44.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:44 smithi131 ceph-mon[41654]: pgmap v767: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:01:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:44 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:01:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:44 smithi131 ceph-mon[41654]: osdmap e500: 8 total, 8 up, 8 in 2024-02-05T08:01:45.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:01:45.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:44 smithi131 ceph-mon[29507]: pgmap v767: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:01:45.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:44 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:45.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:01:45.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:44 smithi131 ceph-mon[29507]: osdmap e500: 8 total, 8 up, 8 in 2024-02-05T08:01:45.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:01:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:44 smithi157 ceph-mon[33345]: pgmap v767: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:01:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:44 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:01:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:44 smithi157 ceph-mon[33345]: osdmap e500: 8 total, 8 up, 8 in 2024-02-05T08:01:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:01:46.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:45 smithi131 ceph-mon[41654]: osdmap e501: 8 total, 8 up, 8 in 2024-02-05T08:01:46.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:45 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:46.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:45 smithi131 ceph-mon[29507]: osdmap e501: 8 total, 8 up, 8 in 2024-02-05T08:01:46.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:45 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:46.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:45 smithi157 ceph-mon[33345]: osdmap e501: 8 total, 8 up, 8 in 2024-02-05T08:01:46.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:45 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:47.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:46 smithi131 ceph-mon[41654]: pgmap v770: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:01:47.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:46 smithi131 ceph-mon[29507]: pgmap v770: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:01:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:46 smithi157 ceph-mon[33345]: pgmap v770: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:01:49.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:48 smithi131 ceph-mon[29507]: pgmap v771: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 887 B/s rd, 177 B/s wr, 1 op/s 2024-02-05T08:01:49.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:48 smithi131 ceph-mon[41654]: pgmap v771: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 887 B/s rd, 177 B/s wr, 1 op/s 2024-02-05T08:01:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:48 smithi157 ceph-mon[33345]: pgmap v771: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 887 B/s rd, 177 B/s wr, 1 op/s 2024-02-05T08:01:49.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:01:48 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:01:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:49 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:50.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:49 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:50.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:49 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:51.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:50 smithi131 ceph-mon[41654]: pgmap v772: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 908 B/s wr, 3 op/s 2024-02-05T08:01:51.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:50 smithi131 ceph-mon[29507]: pgmap v772: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 908 B/s wr, 3 op/s 2024-02-05T08:01:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:50 smithi157 ceph-mon[33345]: pgmap v772: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 908 B/s wr, 3 op/s 2024-02-05T08:01:52.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:51 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:52.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:51 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:52.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:51 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:52 smithi157 ceph-mon[33345]: pgmap v773: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-02-05T08:01:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:52 smithi131 ceph-mon[41654]: pgmap v773: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-02-05T08:01:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:52 smithi131 ceph-mon[29507]: pgmap v773: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-02-05T08:01:53.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:01:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:01:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:01:55.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:54 smithi157 ceph-mon[33345]: pgmap v774: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 525 B/s wr, 2 op/s 2024-02-05T08:01:55.221 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:01:54 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:01:54.781+0000 7f865ed11700 -1 snap_mapper.add_oid found existing snaps mapped on 108:67b79909:test-rados-api-smithi131-82290-97::foo:21, removing 2024-02-05T08:01:55.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:54 smithi131 ceph-mon[29507]: pgmap v774: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 525 B/s wr, 2 op/s 2024-02-05T08:01:55.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:54 smithi131 ceph-mon[41654]: pgmap v774: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 525 B/s wr, 2 op/s 2024-02-05T08:01:55.268 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:01:54 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2[58548]: 2024-02-05T08:01:54.780+0000 7f97532e6700 -1 snap_mapper.add_oid found existing snaps mapped on 108:67b79909:test-rados-api-smithi131-82290-97::foo:21, removing 2024-02-05T08:01:55.268 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:01:54 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3[65147]: 2024-02-05T08:01:54.780+0000 7f18b3388700 -1 snap_mapper.add_oid found existing snaps mapped on 108:67b79909:test-rados-api-smithi131-82290-97::foo:21, removing 2024-02-05T08:01:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-96"}]: dispatch 2024-02-05T08:01:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-96"}]: dispatch 2024-02-05T08:01:56.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-96"}]: dispatch 2024-02-05T08:01:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-96"}]: dispatch 2024-02-05T08:01:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:01:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-96"}]: dispatch 2024-02-05T08:01:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-96"}]: dispatch 2024-02-05T08:01:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:56 smithi157 ceph-mon[33345]: pgmap v775: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 952 B/s rd, 476 B/s wr, 1 op/s 2024-02-05T08:01:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:56 smithi157 ceph-mon[33345]: osdmap e502: 8 total, 8 up, 8 in 2024-02-05T08:01:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:56 smithi131 ceph-mon[41654]: pgmap v775: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 952 B/s rd, 476 B/s wr, 1 op/s 2024-02-05T08:01:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:56 smithi131 ceph-mon[41654]: osdmap e502: 8 total, 8 up, 8 in 2024-02-05T08:01:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:56 smithi131 ceph-mon[29507]: pgmap v775: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 952 B/s rd, 476 B/s wr, 1 op/s 2024-02-05T08:01:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:56 smithi131 ceph-mon[29507]: osdmap e502: 8 total, 8 up, 8 in 2024-02-05T08:01:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:57 smithi157 ceph-mon[33345]: osdmap e503: 8 total, 8 up, 8 in 2024-02-05T08:01:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:57 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:58.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:57 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:58.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:57 smithi131 ceph-mon[41654]: osdmap e503: 8 total, 8 up, 8 in 2024-02-05T08:01:58.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:57 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:58.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:57 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:58.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:57 smithi131 ceph-mon[29507]: osdmap e503: 8 total, 8 up, 8 in 2024-02-05T08:01:58.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:57 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:58.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:57 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:01:58.732 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510479f0 2024-02-05T08:01:58.732 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251047b20 2024-02-05T08:01:58.732 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251047670 2024-02-05T08:01:58.732 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251047c50 2024-02-05T08:01:58.732 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251048460 2024-02-05T08:01:58.732 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251047fc0 2024-02-05T08:01:58.732 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510480d0 2024-02-05T08:01:58.732 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251048200 2024-02-05T08:01:58.732 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251048350 2024-02-05T08:01:58.732 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251048a30 2024-02-05T08:01:58.732 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251048590 2024-02-05T08:01:58.733 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510486c0 2024-02-05T08:01:58.733 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510487d0 2024-02-05T08:01:58.733 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251048920 2024-02-05T08:01:58.733 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251049020 2024-02-05T08:01:58.733 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251048b80 2024-02-05T08:01:58.733 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251048cb0 2024-02-05T08:01:58.733 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251048e00 2024-02-05T08:01:58.733 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251049170 2024-02-05T08:01:58.733 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251049600 2024-02-05T08:01:58.733 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251049730 2024-02-05T08:01:58.733 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251049280 2024-02-05T08:01:58.733 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x5592510493d0 2024-02-05T08:01:58.733 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251049d00 2024-02-05T08:01:58.733 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251049860 2024-02-05T08:01:58.733 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251049990 2024-02-05T08:01:58.734 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251049ac0 2024-02-05T08:01:58.734 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251049bf0 2024-02-05T08:01:58.734 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559250e74600 2024-02-05T08:01:58.734 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559250e74c30 2024-02-05T08:01:58.734 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559250e74790 2024-02-05T08:01:58.734 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559250e748a0 2024-02-05T08:01:58.734 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559250e749f0 2024-02-05T08:01:58.734 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559250e74b20 2024-02-05T08:01:58.734 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559250e75200 2024-02-05T08:01:58.734 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559250e75330 2024-02-05T08:01:58.734 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559250e75460 2024-02-05T08:01:58.734 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559250e755b0 2024-02-05T08:01:58.734 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251049e30 2024-02-05T08:01:58.734 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559250e74d60 2024-02-05T08:01:58.734 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559250e74eb0 2024-02-05T08:01:58.734 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559250e75000 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x559251049f80 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925104a410 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925104a540 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925104a090 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925104a1e0 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925104ab10 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925104a670 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925104a7a0 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55925104a8d0 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.735 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.736 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.736 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.736 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.736 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.736 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.736 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.736 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.736 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.736 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.736 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.736 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.736 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.736 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.736 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.736 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.736 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.737 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.737 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.737 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.737 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.737 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.737 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.737 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.737 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.737 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.737 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.737 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.737 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.737 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:58.737 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:59.122 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:01:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:01:59.122 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:58 smithi157 ceph-mon[33345]: pgmap v778: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:01:59.122 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:58 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:59.122 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:01:59.122 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:58 smithi157 ceph-mon[33345]: osdmap e504: 8 total, 8 up, 8 in 2024-02-05T08:01:59.122 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:01:59.122 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:01:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:58 smithi131 ceph-mon[41654]: pgmap v778: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:01:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:58 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:01:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:58 smithi131 ceph-mon[41654]: osdmap e504: 8 total, 8 up, 8 in 2024-02-05T08:01:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:01:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:01:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:58 smithi131 ceph-mon[29507]: pgmap v778: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:01:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:58 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:01:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:01:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:58 smithi131 ceph-mon[29507]: osdmap e504: 8 total, 8 up, 8 in 2024-02-05T08:01:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:01:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:02:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:02:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:59 smithi157 ceph-mon[33345]: osdmap e505: 8 total, 8 up, 8 in 2024-02-05T08:02:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-98", "mode": "writeback"}]: dispatch 2024-02-05T08:02:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-98", "mode": "writeback"}]: dispatch 2024-02-05T08:02:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:59 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:01:59 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:00.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:02:00.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:59 smithi131 ceph-mon[41654]: osdmap e505: 8 total, 8 up, 8 in 2024-02-05T08:02:00.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-98", "mode": "writeback"}]: dispatch 2024-02-05T08:02:00.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-98", "mode": "writeback"}]: dispatch 2024-02-05T08:02:00.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:59 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:00.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:01:59 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:02:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:59 smithi131 ceph-mon[29507]: osdmap e505: 8 total, 8 up, 8 in 2024-02-05T08:02:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-98", "mode": "writeback"}]: dispatch 2024-02-05T08:02:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-98", "mode": "writeback"}]: dispatch 2024-02-05T08:02:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:59 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:01:59 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:00 smithi157 ceph-mon[33345]: pgmap v781: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-02-05T08:02:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:00 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:02:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-98", "mode": "writeback"}]': finished 2024-02-05T08:02:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-98"}]: dispatch 2024-02-05T08:02:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:00 smithi157 ceph-mon[33345]: osdmap e506: 8 total, 8 up, 8 in 2024-02-05T08:02:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-98"}]: dispatch 2024-02-05T08:02:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:00 smithi131 ceph-mon[41654]: pgmap v781: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-02-05T08:02:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:00 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:02:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-98", "mode": "writeback"}]': finished 2024-02-05T08:02:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-98"}]: dispatch 2024-02-05T08:02:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:00 smithi131 ceph-mon[41654]: osdmap e506: 8 total, 8 up, 8 in 2024-02-05T08:02:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-98"}]: dispatch 2024-02-05T08:02:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:00 smithi131 ceph-mon[29507]: pgmap v781: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-02-05T08:02:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:00 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:02:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-98", "mode": "writeback"}]': finished 2024-02-05T08:02:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-98"}]: dispatch 2024-02-05T08:02:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:00 smithi131 ceph-mon[29507]: osdmap e506: 8 total, 8 up, 8 in 2024-02-05T08:02:01.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-98"}]: dispatch 2024-02-05T08:02:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-98"}]': finished 2024-02-05T08:02:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:02:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:01 smithi157 ceph-mon[33345]: osdmap e507: 8 total, 8 up, 8 in 2024-02-05T08:02:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:02:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:01 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:02:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T08:02:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-05T08:02:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:01 smithi157 ceph-mon[33345]: osdmap e508: 8 total, 8 up, 8 in 2024-02-05T08:02:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-05T08:02:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-98"}]': finished 2024-02-05T08:02:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:02:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[41654]: osdmap e507: 8 total, 8 up, 8 in 2024-02-05T08:02:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:02:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:02:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T08:02:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-05T08:02:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[41654]: osdmap e508: 8 total, 8 up, 8 in 2024-02-05T08:02:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-05T08:02:02.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-98"}]': finished 2024-02-05T08:02:02.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:02:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[29507]: osdmap e507: 8 total, 8 up, 8 in 2024-02-05T08:02:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:02:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:02:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T08:02:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-05T08:02:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[29507]: osdmap e508: 8 total, 8 up, 8 in 2024-02-05T08:02:02.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-05T08:02:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:02 smithi157 ceph-mon[33345]: pgmap v784: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T08:02:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_count","val": "1"}]': finished 2024-02-05T08:02:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:02:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:02 smithi157 ceph-mon[33345]: osdmap e509: 8 total, 8 up, 8 in 2024-02-05T08:02:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:02:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:02:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:02:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:02:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:02 smithi131 ceph-mon[41654]: pgmap v784: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T08:02:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_count","val": "1"}]': finished 2024-02-05T08:02:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:02:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:02 smithi131 ceph-mon[41654]: osdmap e509: 8 total, 8 up, 8 in 2024-02-05T08:02:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:02:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:02 smithi131 ceph-mon[29507]: pgmap v784: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T08:02:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_count","val": "1"}]': finished 2024-02-05T08:02:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:02:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:02 smithi131 ceph-mon[29507]: osdmap e509: 8 total, 8 up, 8 in 2024-02-05T08:02:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:02:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:04 smithi157 ceph-mon[33345]: pgmap v787: 292 pgs: 292 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:02:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:02:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:04 smithi157 ceph-mon[33345]: osdmap e510: 8 total, 8 up, 8 in 2024-02-05T08:02:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-05T08:02:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-05T08:02:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:04 smithi131 ceph-mon[41654]: pgmap v787: 292 pgs: 292 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:02:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:02:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:04 smithi131 ceph-mon[41654]: osdmap e510: 8 total, 8 up, 8 in 2024-02-05T08:02:05.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-05T08:02:05.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-05T08:02:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:04 smithi131 ceph-mon[29507]: pgmap v787: 292 pgs: 292 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:02:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:02:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:04 smithi131 ceph-mon[29507]: osdmap e510: 8 total, 8 up, 8 in 2024-02-05T08:02:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-05T08:02:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-05T08:02:06.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "target_max_objects","val": "250"}]': finished 2024-02-05T08:02:06.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:05 smithi157 ceph-mon[33345]: osdmap e511: 8 total, 8 up, 8 in 2024-02-05T08:02:06.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:06.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:06.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "target_max_objects","val": "250"}]': finished 2024-02-05T08:02:06.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:05 smithi131 ceph-mon[41654]: osdmap e511: 8 total, 8 up, 8 in 2024-02-05T08:02:06.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:06.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:06.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-98","var": "target_max_objects","val": "250"}]': finished 2024-02-05T08:02:06.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:05 smithi131 ceph-mon[29507]: osdmap e511: 8 total, 8 up, 8 in 2024-02-05T08:02:06.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:06.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:06 smithi157 ceph-mon[33345]: pgmap v790: 292 pgs: 292 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:02:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T08:02:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98"}]: dispatch 2024-02-05T08:02:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:06 smithi157 ceph-mon[33345]: osdmap e512: 8 total, 8 up, 8 in 2024-02-05T08:02:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98"}]: dispatch 2024-02-05T08:02:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:06 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:06 smithi131 ceph-mon[41654]: pgmap v790: 292 pgs: 292 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:02:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T08:02:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98"}]: dispatch 2024-02-05T08:02:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:06 smithi131 ceph-mon[41654]: osdmap e512: 8 total, 8 up, 8 in 2024-02-05T08:02:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98"}]: dispatch 2024-02-05T08:02:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:06 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:06 smithi131 ceph-mon[29507]: pgmap v790: 292 pgs: 292 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:02:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T08:02:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98"}]: dispatch 2024-02-05T08:02:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:06 smithi131 ceph-mon[29507]: osdmap e512: 8 total, 8 up, 8 in 2024-02-05T08:02:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98"}]: dispatch 2024-02-05T08:02:07.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:06 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:08.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98"}]': finished 2024-02-05T08:02:08.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:07 smithi157 ceph-mon[33345]: osdmap e513: 8 total, 8 up, 8 in 2024-02-05T08:02:08.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98"}]': finished 2024-02-05T08:02:08.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:07 smithi131 ceph-mon[41654]: osdmap e513: 8 total, 8 up, 8 in 2024-02-05T08:02:08.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-98"}]': finished 2024-02-05T08:02:08.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:07 smithi131 ceph-mon[29507]: osdmap e513: 8 total, 8 up, 8 in 2024-02-05T08:02:09.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:08 smithi157 ceph-mon[33345]: pgmap v793: 292 pgs: 292 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:09.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:08 smithi157 ceph-mon[33345]: osdmap e514: 8 total, 8 up, 8 in 2024-02-05T08:02:09.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:02:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:02:09.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:08 smithi131 ceph-mon[41654]: pgmap v793: 292 pgs: 292 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:09.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:08 smithi131 ceph-mon[41654]: osdmap e514: 8 total, 8 up, 8 in 2024-02-05T08:02:09.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:08 smithi131 ceph-mon[29507]: pgmap v793: 292 pgs: 292 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:09.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:08 smithi131 ceph-mon[29507]: osdmap e514: 8 total, 8 up, 8 in 2024-02-05T08:02:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:09 smithi157 ceph-mon[33345]: osdmap e515: 8 total, 8 up, 8 in 2024-02-05T08:02:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:09 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:10.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:09 smithi131 ceph-mon[41654]: osdmap e515: 8 total, 8 up, 8 in 2024-02-05T08:02:10.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:10.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:10.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:09 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:09 smithi131 ceph-mon[29507]: osdmap e515: 8 total, 8 up, 8 in 2024-02-05T08:02:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:09 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:10 smithi157 ceph-mon[33345]: pgmap v796: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:02:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:10 smithi157 ceph-mon[33345]: osdmap e516: 8 total, 8 up, 8 in 2024-02-05T08:02:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:02:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:02:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:10 smithi131 ceph-mon[29507]: pgmap v796: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:11.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:02:11.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:10 smithi131 ceph-mon[29507]: osdmap e516: 8 total, 8 up, 8 in 2024-02-05T08:02:11.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:02:11.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:02:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:10 smithi131 ceph-mon[41654]: pgmap v796: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:02:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:10 smithi131 ceph-mon[41654]: osdmap e516: 8 total, 8 up, 8 in 2024-02-05T08:02:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:02:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:02:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:02:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:11 smithi157 ceph-mon[33345]: osdmap e517: 8 total, 8 up, 8 in 2024-02-05T08:02:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:11 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-100"}]: dispatch 2024-02-05T08:02:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-100"}]: dispatch 2024-02-05T08:02:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:11 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-100"}]': finished 2024-02-05T08:02:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:11 smithi157 ceph-mon[33345]: osdmap e518: 8 total, 8 up, 8 in 2024-02-05T08:02:12.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:02:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:11 smithi131 ceph-mon[41654]: osdmap e517: 8 total, 8 up, 8 in 2024-02-05T08:02:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:11 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-100"}]: dispatch 2024-02-05T08:02:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-100"}]: dispatch 2024-02-05T08:02:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:11 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-100"}]': finished 2024-02-05T08:02:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:11 smithi131 ceph-mon[41654]: osdmap e518: 8 total, 8 up, 8 in 2024-02-05T08:02:12.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:02:12.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:11 smithi131 ceph-mon[29507]: osdmap e517: 8 total, 8 up, 8 in 2024-02-05T08:02:12.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:11 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-100"}]: dispatch 2024-02-05T08:02:12.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-100"}]: dispatch 2024-02-05T08:02:12.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:11 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:12.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-6", "overlaypool": "test-rados-api-smithi131-82290-100"}]': finished 2024-02-05T08:02:12.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:11 smithi131 ceph-mon[29507]: osdmap e518: 8 total, 8 up, 8 in 2024-02-05T08:02:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:12 smithi157 ceph-mon[33345]: pgmap v799: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-100", "mode": "writeback"}]: dispatch 2024-02-05T08:02:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-100", "mode": "writeback"}]: dispatch 2024-02-05T08:02:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:12 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:02:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-100", "mode": "writeback"}]': finished 2024-02-05T08:02:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:02:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:12 smithi157 ceph-mon[33345]: osdmap e519: 8 total, 8 up, 8 in 2024-02-05T08:02:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:02:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:02:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:02:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-mon[41654]: pgmap v799: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-100", "mode": "writeback"}]: dispatch 2024-02-05T08:02:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-100", "mode": "writeback"}]: dispatch 2024-02-05T08:02:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:02:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-100", "mode": "writeback"}]': finished 2024-02-05T08:02:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:02:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-mon[41654]: osdmap e519: 8 total, 8 up, 8 in 2024-02-05T08:02:13.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:02:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-mon[29507]: pgmap v799: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-100", "mode": "writeback"}]: dispatch 2024-02-05T08:02:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-100", "mode": "writeback"}]: dispatch 2024-02-05T08:02:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:02:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-100", "mode": "writeback"}]': finished 2024-02-05T08:02:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:02:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-mon[29507]: osdmap e519: 8 total, 8 up, 8 in 2024-02-05T08:02:13.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:02:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:14 smithi157 ceph-mon[33345]: pgmap v802: 292 pgs: 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_count","val": "2"}]': finished 2024-02-05T08:02:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:14 smithi157 ceph-mon[33345]: osdmap e520: 8 total, 8 up, 8 in 2024-02-05T08:02:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:02:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:02:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:14 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:02:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:14 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:15.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:14 smithi131 ceph-mon[41654]: pgmap v802: 292 pgs: 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_count","val": "2"}]': finished 2024-02-05T08:02:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:14 smithi131 ceph-mon[41654]: osdmap e520: 8 total, 8 up, 8 in 2024-02-05T08:02:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:02:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:02:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:14 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:02:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:14 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:14 smithi131 ceph-mon[29507]: pgmap v802: 292 pgs: 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_count","val": "2"}]': finished 2024-02-05T08:02:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:14 smithi131 ceph-mon[29507]: osdmap e520: 8 total, 8 up, 8 in 2024-02-05T08:02:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:02:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:02:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:14 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:02:15.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:14 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:02:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T08:02:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:15 smithi157 ceph-mon[33345]: osdmap e521: 8 total, 8 up, 8 in 2024-02-05T08:02:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T08:02:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:02:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:15 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:02:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T08:02:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-05T08:02:16.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:15 smithi157 ceph-mon[33345]: osdmap e522: 8 total, 8 up, 8 in 2024-02-05T08:02:16.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-05T08:02:16.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:02:16.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T08:02:16.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[41654]: osdmap e521: 8 total, 8 up, 8 in 2024-02-05T08:02:16.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T08:02:16.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:02:16.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:02:16.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T08:02:16.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-05T08:02:16.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[41654]: osdmap e522: 8 total, 8 up, 8 in 2024-02-05T08:02:16.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-05T08:02:16.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:02:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T08:02:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[29507]: osdmap e521: 8 total, 8 up, 8 in 2024-02-05T08:02:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T08:02:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:02:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:02:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T08:02:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-05T08:02:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[29507]: osdmap e522: 8 total, 8 up, 8 in 2024-02-05T08:02:16.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-05T08:02:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:16 smithi157 ceph-mon[33345]: pgmap v805: 292 pgs: 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:02:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:02:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:16 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:02:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:16 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-05T08:02:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:16 smithi157 ceph-mon[33345]: osdmap e523: 8 total, 8 up, 8 in 2024-02-05T08:02:17.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:16 smithi131 ceph-mon[41654]: pgmap v805: 292 pgs: 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:17.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:02:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:02:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:16 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:02:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:16 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-05T08:02:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:16 smithi131 ceph-mon[41654]: osdmap e523: 8 total, 8 up, 8 in 2024-02-05T08:02:17.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:16 smithi131 ceph-mon[29507]: pgmap v805: 292 pgs: 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:17.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:02:17.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:02:17.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:16 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:02:17.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:16 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:17.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-05T08:02:17.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:16 smithi131 ceph-mon[29507]: osdmap e523: 8 total, 8 up, 8 in 2024-02-05T08:02:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:18.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:18.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:18.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:18.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:19.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:02:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:02:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:18 smithi157 ceph-mon[33345]: pgmap v808: 292 pgs: 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T08:02:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100"}]: dispatch 2024-02-05T08:02:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:18 smithi157 ceph-mon[33345]: osdmap e524: 8 total, 8 up, 8 in 2024-02-05T08:02:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100"}]: dispatch 2024-02-05T08:02:19.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:18 smithi131 ceph-mon[41654]: pgmap v808: 292 pgs: 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:19.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T08:02:19.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100"}]: dispatch 2024-02-05T08:02:19.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:18 smithi131 ceph-mon[41654]: osdmap e524: 8 total, 8 up, 8 in 2024-02-05T08:02:19.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100"}]: dispatch 2024-02-05T08:02:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:18 smithi131 ceph-mon[29507]: pgmap v808: 292 pgs: 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]': finished 2024-02-05T08:02:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100"}]: dispatch 2024-02-05T08:02:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:18 smithi131 ceph-mon[29507]: osdmap e524: 8 total, 8 up, 8 in 2024-02-05T08:02:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100"}]: dispatch 2024-02-05T08:02:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:19 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100"}]': finished 2024-02-05T08:02:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:19 smithi157 ceph-mon[33345]: osdmap e525: 8 total, 8 up, 8 in 2024-02-05T08:02:20.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:19 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:20.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100"}]': finished 2024-02-05T08:02:20.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:19 smithi131 ceph-mon[41654]: osdmap e525: 8 total, 8 up, 8 in 2024-02-05T08:02:20.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:19 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:20.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-100"}]': finished 2024-02-05T08:02:20.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:19 smithi131 ceph-mon[29507]: osdmap e525: 8 total, 8 up, 8 in 2024-02-05T08:02:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:20 smithi157 ceph-mon[33345]: pgmap v811: 292 pgs: 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T08:02:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:20 smithi157 ceph-mon[33345]: osdmap e526: 8 total, 8 up, 8 in 2024-02-05T08:02:21.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:20 smithi131 ceph-mon[29507]: pgmap v811: 292 pgs: 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T08:02:21.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:20 smithi131 ceph-mon[29507]: osdmap e526: 8 total, 8 up, 8 in 2024-02-05T08:02:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:20 smithi131 ceph-mon[41654]: pgmap v811: 292 pgs: 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T08:02:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:20 smithi131 ceph-mon[41654]: osdmap e526: 8 total, 8 up, 8 in 2024-02-05T08:02:22.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:21 smithi157 ceph-mon[33345]: osdmap e527: 8 total, 8 up, 8 in 2024-02-05T08:02:22.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:22.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:22.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:21 smithi131 ceph-mon[41654]: osdmap e527: 8 total, 8 up, 8 in 2024-02-05T08:02:22.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:22.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:22.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:21 smithi131 ceph-mon[29507]: osdmap e527: 8 total, 8 up, 8 in 2024-02-05T08:02:22.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:22.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:22 smithi157 ceph-mon[33345]: pgmap v814: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:02:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:22 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:02:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:22 smithi157 ceph-mon[33345]: osdmap e528: 8 total, 8 up, 8 in 2024-02-05T08:02:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:02:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:02:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:02:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:02:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:02:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:02:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:22 smithi131 ceph-mon[41654]: pgmap v814: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:02:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:22 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:02:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:22 smithi131 ceph-mon[41654]: osdmap e528: 8 total, 8 up, 8 in 2024-02-05T08:02:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:02:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:02:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:02:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:22 smithi131 ceph-mon[29507]: pgmap v814: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:02:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:22 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:02:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:22 smithi131 ceph-mon[29507]: osdmap e528: 8 total, 8 up, 8 in 2024-02-05T08:02:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T08:02:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:02:23.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T08:02:24.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:02:24.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:02:24.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:23 smithi157 ceph-mon[33345]: osdmap e529: 8 total, 8 up, 8 in 2024-02-05T08:02:24.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:02:24.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:02:24.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:02:24.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:23 smithi131 ceph-mon[41654]: osdmap e529: 8 total, 8 up, 8 in 2024-02-05T08:02:24.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:02:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T08:02:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:02:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:23 smithi131 ceph-mon[29507]: osdmap e529: 8 total, 8 up, 8 in 2024-02-05T08:02:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T08:02:24.965 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:24 smithi131 ceph-mon[41654]: pgmap v817: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:24.965 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T08:02:24.965 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:24 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:02:24.965 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:24 smithi131 ceph-mon[29507]: pgmap v817: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:24.966 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T08:02:24.966 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:02:24.966 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:24 smithi131 ceph-mon[29507]: osdmap e530: 8 total, 8 up, 8 in 2024-02-05T08:02:24.966 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:02:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:24 smithi157 ceph-mon[33345]: pgmap v817: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T08:02:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:24 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:02:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:24 smithi157 ceph-mon[33345]: osdmap e530: 8 total, 8 up, 8 in 2024-02-05T08:02:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:02:25.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:24 smithi131 ceph-mon[41654]: osdmap e530: 8 total, 8 up, 8 in 2024-02-05T08:02:25.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T08:02:26.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:02:26.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:25 smithi157 ceph-mon[33345]: osdmap e531: 8 total, 8 up, 8 in 2024-02-05T08:02:26.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:26.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:26.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-102"}]: dispatch 2024-02-05T08:02:26.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-102"}]: dispatch 2024-02-05T08:02:26.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:02:26.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:25 smithi131 ceph-mon[41654]: osdmap e531: 8 total, 8 up, 8 in 2024-02-05T08:02:26.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:26.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:26.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-102"}]: dispatch 2024-02-05T08:02:26.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-102"}]: dispatch 2024-02-05T08:02:26.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T08:02:26.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:25 smithi131 ceph-mon[29507]: osdmap e531: 8 total, 8 up, 8 in 2024-02-05T08:02:26.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:26.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-6"}]: dispatch 2024-02-05T08:02:26.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3811755407' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-102"}]: dispatch 2024-02-05T08:02:26.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-6", "tierpool": "test-rados-api-smithi131-82290-102"}]: dispatch 2024-02-05T08:02:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:26 smithi157 ceph-mon[33345]: pgmap v820: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:26 smithi157 ceph-mon[33345]: osdmap e532: 8 total, 8 up, 8 in 2024-02-05T08:02:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:26 smithi131 ceph-mon[41654]: pgmap v820: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:26 smithi131 ceph-mon[41654]: osdmap e532: 8 total, 8 up, 8 in 2024-02-05T08:02:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:26 smithi131 ceph-mon[29507]: pgmap v820: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:26 smithi131 ceph-mon[29507]: osdmap e532: 8 total, 8 up, 8 in 2024-02-05T08:02:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:27 smithi131 ceph-mon[41654]: osdmap e533: 8 total, 8 up, 8 in 2024-02-05T08:02:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:27 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:27 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:28.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:27 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T08:02:28.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T08:02:28.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:27 smithi131 ceph-mon[29507]: osdmap e533: 8 total, 8 up, 8 in 2024-02-05T08:02:28.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:27 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:28.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:28.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:27 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:28.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:28.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:27 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T08:02:28.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T08:02:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:27 smithi157 ceph-mon[33345]: osdmap e533: 8 total, 8 up, 8 in 2024-02-05T08:02:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:27 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:27 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:27 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T08:02:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T08:02:29.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:02:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:02:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:29 smithi131 ceph-mon[41654]: pgmap v823: 228 pgs: 228 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:29.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T08:02:29.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:29 smithi131 ceph-mon[41654]: osdmap e534: 8 total, 8 up, 8 in 2024-02-05T08:02:29.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi131-82290-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:29.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi131-82290-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:29.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:29 smithi131 ceph-mon[29507]: pgmap v823: 228 pgs: 228 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:29.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T08:02:29.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:29 smithi131 ceph-mon[29507]: osdmap e534: 8 total, 8 up, 8 in 2024-02-05T08:02:29.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi131-82290-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:29.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi131-82290-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:29.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:29 smithi157 ceph-mon[33345]: pgmap v823: 228 pgs: 228 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:29.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T08:02:29.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:29 smithi157 ceph-mon[33345]: osdmap e534: 8 total, 8 up, 8 in 2024-02-05T08:02:29.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi131-82290-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:29.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi131-82290-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:02:30.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:30 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:30.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:30 smithi131 ceph-mon[41654]: osdmap e535: 8 total, 8 up, 8 in 2024-02-05T08:02:30.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:30 smithi131 ceph-mon[41654]: pgmap v826: 228 pgs: 228 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:30.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:30 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:02:30.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:30 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:30.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi131-82290-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi131-82290-104"}]': finished 2024-02-05T08:02:30.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:30 smithi131 ceph-mon[41654]: osdmap e536: 8 total, 8 up, 8 in 2024-02-05T08:02:30.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:30 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:30.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:30 smithi131 ceph-mon[29507]: osdmap e535: 8 total, 8 up, 8 in 2024-02-05T08:02:30.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:30 smithi131 ceph-mon[29507]: pgmap v826: 228 pgs: 228 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:30.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:30 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:02:30.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:30 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:30.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi131-82290-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi131-82290-104"}]': finished 2024-02-05T08:02:30.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:30 smithi131 ceph-mon[29507]: osdmap e536: 8 total, 8 up, 8 in 2024-02-05T08:02:30.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:30 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:30.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:30 smithi157 ceph-mon[33345]: osdmap e535: 8 total, 8 up, 8 in 2024-02-05T08:02:30.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:30 smithi157 ceph-mon[33345]: pgmap v826: 228 pgs: 228 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:30.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:30 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:02:30.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:30 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:30.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi131-82290-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi131-82290-104"}]': finished 2024-02-05T08:02:30.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:30 smithi157 ceph-mon[33345]: osdmap e536: 8 total, 8 up, 8 in 2024-02-05T08:02:32.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:31 smithi131 ceph-mon[41654]: osdmap e537: 8 total, 8 up, 8 in 2024-02-05T08:02:32.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:31 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:32.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:31 smithi131 ceph-mon[29507]: osdmap e537: 8 total, 8 up, 8 in 2024-02-05T08:02:32.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:31 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:31 smithi157 ceph-mon[33345]: osdmap e537: 8 total, 8 up, 8 in 2024-02-05T08:02:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:31 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:02:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:02:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:02:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:33 smithi131 ceph-mon[41654]: pgmap v829: 236 pgs: 8 unknown, 228 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:33 smithi131 ceph-mon[41654]: osdmap e538: 8 total, 8 up, 8 in 2024-02-05T08:02:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:33 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:33 smithi131 ceph-mon[29507]: pgmap v829: 236 pgs: 8 unknown, 228 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:33 smithi131 ceph-mon[29507]: osdmap e538: 8 total, 8 up, 8 in 2024-02-05T08:02:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:33 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:33 smithi157 ceph-mon[33345]: pgmap v829: 236 pgs: 8 unknown, 228 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:33 smithi157 ceph-mon[33345]: osdmap e538: 8 total, 8 up, 8 in 2024-02-05T08:02:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:33 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:34.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:02:34.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:34 smithi131 ceph-mon[41654]: osdmap e539: 8 total, 8 up, 8 in 2024-02-05T08:02:34.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:34 smithi131 ceph-mon[41654]: osdmap e540: 8 total, 8 up, 8 in 2024-02-05T08:02:34.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:34.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:34.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:02:34.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:34 smithi131 ceph-mon[29507]: osdmap e539: 8 total, 8 up, 8 in 2024-02-05T08:02:34.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:34 smithi131 ceph-mon[29507]: osdmap e540: 8 total, 8 up, 8 in 2024-02-05T08:02:34.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:34.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:34.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:02:34.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:34 smithi157 ceph-mon[33345]: osdmap e539: 8 total, 8 up, 8 in 2024-02-05T08:02:34.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:34 smithi157 ceph-mon[33345]: osdmap e540: 8 total, 8 up, 8 in 2024-02-05T08:02:34.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:34.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:35 smithi131 ceph-mon[41654]: pgmap v832: 268 pgs: 268 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:02:35.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:02:35.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:35 smithi131 ceph-mon[41654]: osdmap e541: 8 total, 8 up, 8 in 2024-02-05T08:02:35.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:35.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:35 smithi131 ceph-mon[29507]: pgmap v832: 268 pgs: 268 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:02:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:02:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:35 smithi131 ceph-mon[29507]: osdmap e541: 8 total, 8 up, 8 in 2024-02-05T08:02:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:35 smithi157 ceph-mon[33345]: pgmap v832: 268 pgs: 268 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:02:35.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:02:35.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:35 smithi157 ceph-mon[33345]: osdmap e541: 8 total, 8 up, 8 in 2024-02-05T08:02:35.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:35.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:37 smithi131 ceph-mon[41654]: pgmap v835: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:02:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]': finished 2024-02-05T08:02:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:37 smithi131 ceph-mon[41654]: osdmap e542: 8 total, 8 up, 8 in 2024-02-05T08:02:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-107", "overlaypool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-107", "overlaypool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:37.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:37 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:37 smithi131 ceph-mon[29507]: pgmap v835: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:02:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]': finished 2024-02-05T08:02:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:37 smithi131 ceph-mon[29507]: osdmap e542: 8 total, 8 up, 8 in 2024-02-05T08:02:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-107", "overlaypool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-107", "overlaypool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:37.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:37 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:37.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:37 smithi157 ceph-mon[33345]: pgmap v835: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:02:37.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]': finished 2024-02-05T08:02:37.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:37 smithi157 ceph-mon[33345]: osdmap e542: 8 total, 8 up, 8 in 2024-02-05T08:02:37.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-107", "overlaypool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:37.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-107", "overlaypool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:37.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:37 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:38.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-107", "overlaypool": "test-rados-api-smithi131-82290-107-cache"}]': finished 2024-02-05T08:02:38.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:38 smithi131 ceph-mon[41654]: osdmap e543: 8 total, 8 up, 8 in 2024-02-05T08:02:38.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:38 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-107-cache", "mode": "writeback"}]: dispatch 2024-02-05T08:02:38.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-107-cache", "mode": "writeback"}]: dispatch 2024-02-05T08:02:38.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-107", "overlaypool": "test-rados-api-smithi131-82290-107-cache"}]': finished 2024-02-05T08:02:38.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:38 smithi131 ceph-mon[29507]: osdmap e543: 8 total, 8 up, 8 in 2024-02-05T08:02:38.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:38 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-107-cache", "mode": "writeback"}]: dispatch 2024-02-05T08:02:38.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-107-cache", "mode": "writeback"}]: dispatch 2024-02-05T08:02:38.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-107", "overlaypool": "test-rados-api-smithi131-82290-107-cache"}]': finished 2024-02-05T08:02:38.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:38 smithi157 ceph-mon[33345]: osdmap e543: 8 total, 8 up, 8 in 2024-02-05T08:02:38.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:38 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-107-cache", "mode": "writeback"}]: dispatch 2024-02-05T08:02:38.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-107-cache", "mode": "writeback"}]: dispatch 2024-02-05T08:02:39.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:02:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:02:39.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:39 smithi157 ceph-mon[33345]: pgmap v838: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:39.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:39 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:02:39.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-107-cache", "mode": "writeback"}]': finished 2024-02-05T08:02:39.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:39 smithi157 ceph-mon[33345]: osdmap e544: 8 total, 8 up, 8 in 2024-02-05T08:02:39.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:39 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-107"}]: dispatch 2024-02-05T08:02:39.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-107"}]: dispatch 2024-02-05T08:02:39.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:39 smithi131 ceph-mon[41654]: pgmap v838: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:39.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:39 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:02:39.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-107-cache", "mode": "writeback"}]': finished 2024-02-05T08:02:39.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:39 smithi131 ceph-mon[41654]: osdmap e544: 8 total, 8 up, 8 in 2024-02-05T08:02:39.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:39 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-107"}]: dispatch 2024-02-05T08:02:39.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-107"}]: dispatch 2024-02-05T08:02:39.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:39 smithi131 ceph-mon[29507]: pgmap v838: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:39.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:39 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:02:39.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-107-cache", "mode": "writeback"}]': finished 2024-02-05T08:02:39.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:39 smithi131 ceph-mon[29507]: osdmap e544: 8 total, 8 up, 8 in 2024-02-05T08:02:39.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-107"}]: dispatch 2024-02-05T08:02:39.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-107"}]: dispatch 2024-02-05T08:02:40.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:40 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:40.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-107"}]': finished 2024-02-05T08:02:40.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:40 smithi157 ceph-mon[33345]: osdmap e545: 8 total, 8 up, 8 in 2024-02-05T08:02:40.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:40 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:40.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:40.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:40 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:02:40.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]': finished 2024-02-05T08:02:40.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:40 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:40.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-107"}]': finished 2024-02-05T08:02:40.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:40 smithi131 ceph-mon[29507]: osdmap e545: 8 total, 8 up, 8 in 2024-02-05T08:02:40.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:40 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:40.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:40.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:40 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:02:40.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]': finished 2024-02-05T08:02:40.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:40 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:40.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-107"}]': finished 2024-02-05T08:02:40.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:40 smithi131 ceph-mon[41654]: osdmap e545: 8 total, 8 up, 8 in 2024-02-05T08:02:40.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:40 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2149136128' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:40.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]: dispatch 2024-02-05T08:02:40.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:40 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:02:40.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-107", "tierpool": "test-rados-api-smithi131-82290-107-cache"}]': finished 2024-02-05T08:02:41.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:41 smithi157 ceph-mon[33345]: pgmap v841: 300 pgs: 300 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:02:41.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:41 smithi157 ceph-mon[33345]: osdmap e546: 8 total, 8 up, 8 in 2024-02-05T08:02:41.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:41 smithi157 ceph-mon[33345]: osdmap e547: 8 total, 8 up, 8 in 2024-02-05T08:02:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:41 smithi131 ceph-mon[41654]: pgmap v841: 300 pgs: 300 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:02:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:41 smithi131 ceph-mon[41654]: osdmap e546: 8 total, 8 up, 8 in 2024-02-05T08:02:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:41 smithi131 ceph-mon[41654]: osdmap e547: 8 total, 8 up, 8 in 2024-02-05T08:02:41.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:41 smithi131 ceph-mon[29507]: pgmap v841: 300 pgs: 300 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:02:41.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:41 smithi131 ceph-mon[29507]: osdmap e546: 8 total, 8 up, 8 in 2024-02-05T08:02:41.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:41 smithi131 ceph-mon[29507]: osdmap e547: 8 total, 8 up, 8 in 2024-02-05T08:02:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:02:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:02:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:02:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:43 smithi131 ceph-mon[41654]: pgmap v844: 268 pgs: 268 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:02:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:43 smithi131 ceph-mon[41654]: osdmap e548: 8 total, 8 up, 8 in 2024-02-05T08:02:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82290-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T08:02:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:43 smithi131 ceph-mon[29507]: pgmap v844: 268 pgs: 268 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:02:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:43 smithi131 ceph-mon[29507]: osdmap e548: 8 total, 8 up, 8 in 2024-02-05T08:02:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:43.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82290-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T08:02:43.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:43 smithi157 ceph-mon[33345]: pgmap v844: 268 pgs: 268 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:02:43.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:43 smithi157 ceph-mon[33345]: osdmap e548: 8 total, 8 up, 8 in 2024-02-05T08:02:43.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:43.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:43.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82290-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T08:02:44.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82290-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T08:02:44.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:44 smithi157 ceph-mon[33345]: osdmap e549: 8 total, 8 up, 8 in 2024-02-05T08:02:44.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82290-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:44.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82290-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T08:02:44.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:44 smithi131 ceph-mon[41654]: osdmap e549: 8 total, 8 up, 8 in 2024-02-05T08:02:44.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82290-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:44.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82290-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T08:02:44.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:44 smithi131 ceph-mon[29507]: osdmap e549: 8 total, 8 up, 8 in 2024-02-05T08:02:44.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82290-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:45 smithi157 ceph-mon[33345]: pgmap v847: 236 pgs: 236 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:45 smithi157 ceph-mon[33345]: osdmap e550: 8 total, 8 up, 8 in 2024-02-05T08:02:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:45 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:45 smithi131 ceph-mon[41654]: pgmap v847: 236 pgs: 236 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:45 smithi131 ceph-mon[41654]: osdmap e550: 8 total, 8 up, 8 in 2024-02-05T08:02:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:45 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:45 smithi131 ceph-mon[29507]: pgmap v847: 236 pgs: 236 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:45 smithi131 ceph-mon[29507]: osdmap e550: 8 total, 8 up, 8 in 2024-02-05T08:02:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:45 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:46.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:46 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82290-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82290-109"}]': finished 2024-02-05T08:02:46.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:46 smithi157 ceph-mon[33345]: osdmap e551: 8 total, 8 up, 8 in 2024-02-05T08:02:46.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:46 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82290-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82290-109"}]': finished 2024-02-05T08:02:46.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:46 smithi131 ceph-mon[41654]: osdmap e551: 8 total, 8 up, 8 in 2024-02-05T08:02:46.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:46 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82290-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82290-109"}]': finished 2024-02-05T08:02:46.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:46 smithi131 ceph-mon[29507]: osdmap e551: 8 total, 8 up, 8 in 2024-02-05T08:02:47.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:47 smithi157 ceph-mon[33345]: pgmap v850: 244 pgs: 8 unknown, 236 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:47.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:47 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:47.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:47 smithi157 ceph-mon[33345]: osdmap e552: 8 total, 8 up, 8 in 2024-02-05T08:02:47.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:47 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:47.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:47 smithi131 ceph-mon[41654]: pgmap v850: 244 pgs: 8 unknown, 236 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:47.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:47 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:47.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:47 smithi131 ceph-mon[41654]: osdmap e552: 8 total, 8 up, 8 in 2024-02-05T08:02:47.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:47 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:47.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:47 smithi131 ceph-mon[29507]: pgmap v850: 244 pgs: 8 unknown, 236 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:47.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:47 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:02:47.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:47 smithi131 ceph-mon[29507]: osdmap e552: 8 total, 8 up, 8 in 2024-02-05T08:02:47.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:47 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:02:48.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:02:48.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:48 smithi157 ceph-mon[33345]: osdmap e553: 8 total, 8 up, 8 in 2024-02-05T08:02:48.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-109", "tierpool": "test-rados-api-smithi131-82290-109-cache"}]: dispatch 2024-02-05T08:02:48.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:02:48.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:48 smithi131 ceph-mon[41654]: osdmap e553: 8 total, 8 up, 8 in 2024-02-05T08:02:48.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-109", "tierpool": "test-rados-api-smithi131-82290-109-cache"}]: dispatch 2024-02-05T08:02:48.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:02:48.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:48 smithi131 ceph-mon[29507]: osdmap e553: 8 total, 8 up, 8 in 2024-02-05T08:02:48.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-109", "tierpool": "test-rados-api-smithi131-82290-109-cache"}]: dispatch 2024-02-05T08:02:49.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:02:48 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:02:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:49 smithi157 ceph-mon[33345]: pgmap v853: 276 pgs: 40 unknown, 236 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-109", "tierpool": "test-rados-api-smithi131-82290-109-cache"}]': finished 2024-02-05T08:02:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:49 smithi157 ceph-mon[33345]: osdmap e554: 8 total, 8 up, 8 in 2024-02-05T08:02:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-109", "overlaypool": "test-rados-api-smithi131-82290-109-cache"}]: dispatch 2024-02-05T08:02:49.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:49 smithi131 ceph-mon[41654]: pgmap v853: 276 pgs: 40 unknown, 236 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:49.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-109", "tierpool": "test-rados-api-smithi131-82290-109-cache"}]': finished 2024-02-05T08:02:49.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:49 smithi131 ceph-mon[41654]: osdmap e554: 8 total, 8 up, 8 in 2024-02-05T08:02:49.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-109", "overlaypool": "test-rados-api-smithi131-82290-109-cache"}]: dispatch 2024-02-05T08:02:49.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:49 smithi131 ceph-mon[29507]: pgmap v853: 276 pgs: 40 unknown, 236 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:49.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-109", "tierpool": "test-rados-api-smithi131-82290-109-cache"}]': finished 2024-02-05T08:02:49.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:49 smithi131 ceph-mon[29507]: osdmap e554: 8 total, 8 up, 8 in 2024-02-05T08:02:49.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-109", "overlaypool": "test-rados-api-smithi131-82290-109-cache"}]: dispatch 2024-02-05T08:02:50.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:50 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:50.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-109", "overlaypool": "test-rados-api-smithi131-82290-109-cache"}]': finished 2024-02-05T08:02:50.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:50 smithi157 ceph-mon[33345]: osdmap e555: 8 total, 8 up, 8 in 2024-02-05T08:02:50.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-109-cache", "mode": "writeback"}]: dispatch 2024-02-05T08:02:50.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:50 smithi157 ceph-mon[33345]: pgmap v856: 276 pgs: 276 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:50.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:50 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:02:50.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-109-cache", "mode": "writeback"}]': finished 2024-02-05T08:02:50.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:50 smithi157 ceph-mon[33345]: osdmap e556: 8 total, 8 up, 8 in 2024-02-05T08:02:50.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:02:50.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:50.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-109", "overlaypool": "test-rados-api-smithi131-82290-109-cache"}]': finished 2024-02-05T08:02:50.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[41654]: osdmap e555: 8 total, 8 up, 8 in 2024-02-05T08:02:50.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-109-cache", "mode": "writeback"}]: dispatch 2024-02-05T08:02:50.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[41654]: pgmap v856: 276 pgs: 276 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:50.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:02:50.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-109-cache", "mode": "writeback"}]': finished 2024-02-05T08:02:50.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[41654]: osdmap e556: 8 total, 8 up, 8 in 2024-02-05T08:02:50.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:02:50.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:50.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-109", "overlaypool": "test-rados-api-smithi131-82290-109-cache"}]': finished 2024-02-05T08:02:50.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[29507]: osdmap e555: 8 total, 8 up, 8 in 2024-02-05T08:02:50.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-109-cache", "mode": "writeback"}]: dispatch 2024-02-05T08:02:50.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[29507]: pgmap v856: 276 pgs: 276 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:50.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:02:50.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-109-cache", "mode": "writeback"}]': finished 2024-02-05T08:02:50.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[29507]: osdmap e556: 8 total, 8 up, 8 in 2024-02-05T08:02:50.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:02:52.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-05T08:02:52.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:52 smithi157 ceph-mon[33345]: osdmap e557: 8 total, 8 up, 8 in 2024-02-05T08:02:52.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:02:52.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-05T08:02:52.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:52 smithi131 ceph-mon[41654]: osdmap e557: 8 total, 8 up, 8 in 2024-02-05T08:02:52.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:02:52.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-05T08:02:52.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:52 smithi131 ceph-mon[29507]: osdmap e557: 8 total, 8 up, 8 in 2024-02-05T08:02:52.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:02:53.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:02:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:02:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:02:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:53 smithi131 ceph-mon[29507]: pgmap v859: 276 pgs: 276 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:02:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:53 smithi131 ceph-mon[29507]: osdmap e558: 8 total, 8 up, 8 in 2024-02-05T08:02:53.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T08:02:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:53 smithi131 ceph-mon[41654]: pgmap v859: 276 pgs: 276 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:02:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:53 smithi131 ceph-mon[41654]: osdmap e558: 8 total, 8 up, 8 in 2024-02-05T08:02:53.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T08:02:53.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:53 smithi157 ceph-mon[33345]: pgmap v859: 276 pgs: 276 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:53.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:02:53.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:53 smithi157 ceph-mon[33345]: osdmap e558: 8 total, 8 up, 8 in 2024-02-05T08:02:53.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T08:02:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:54 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:02:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T08:02:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:54 smithi157 ceph-mon[33345]: osdmap e559: 8 total, 8 up, 8 in 2024-02-05T08:02:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-05T08:02:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:54 smithi157 ceph-mon[33345]: pgmap v862: 276 pgs: 276 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:54.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-05T08:02:54.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:54 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:02:54.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T08:02:54.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:54 smithi131 ceph-mon[41654]: osdmap e559: 8 total, 8 up, 8 in 2024-02-05T08:02:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-05T08:02:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:54 smithi131 ceph-mon[41654]: pgmap v862: 276 pgs: 276 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:54.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-05T08:02:54.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:54 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:02:54.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T08:02:54.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:54 smithi131 ceph-mon[29507]: osdmap e559: 8 total, 8 up, 8 in 2024-02-05T08:02:54.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-05T08:02:54.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:54 smithi131 ceph-mon[29507]: pgmap v862: 276 pgs: 276 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:54.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-05T08:02:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:55 smithi157 ceph-mon[33345]: osdmap e560: 8 total, 8 up, 8 in 2024-02-05T08:02:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:55 smithi131 ceph-mon[41654]: osdmap e560: 8 total, 8 up, 8 in 2024-02-05T08:02:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:55.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:55 smithi131 ceph-mon[29507]: osdmap e560: 8 total, 8 up, 8 in 2024-02-05T08:02:55.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-109"}]': finished 2024-02-05T08:02:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:56 smithi157 ceph-mon[33345]: osdmap e561: 8 total, 8 up, 8 in 2024-02-05T08:02:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-109", "tierpool": "test-rados-api-smithi131-82290-109-cache"}]: dispatch 2024-02-05T08:02:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:56 smithi157 ceph-mon[33345]: pgmap v865: 276 pgs: 276 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-109", "tierpool": "test-rados-api-smithi131-82290-109-cache"}]': finished 2024-02-05T08:02:56.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-109"}]': finished 2024-02-05T08:02:56.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:56 smithi131 ceph-mon[41654]: osdmap e561: 8 total, 8 up, 8 in 2024-02-05T08:02:56.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-109", "tierpool": "test-rados-api-smithi131-82290-109-cache"}]: dispatch 2024-02-05T08:02:56.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:56 smithi131 ceph-mon[41654]: pgmap v865: 276 pgs: 276 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:56.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-109", "tierpool": "test-rados-api-smithi131-82290-109-cache"}]': finished 2024-02-05T08:02:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-109"}]': finished 2024-02-05T08:02:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:56 smithi131 ceph-mon[29507]: osdmap e561: 8 total, 8 up, 8 in 2024-02-05T08:02:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-109", "tierpool": "test-rados-api-smithi131-82290-109-cache"}]: dispatch 2024-02-05T08:02:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:56 smithi131 ceph-mon[29507]: pgmap v865: 276 pgs: 276 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-109", "tierpool": "test-rados-api-smithi131-82290-109-cache"}]': finished 2024-02-05T08:02:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:57 smithi157 ceph-mon[33345]: osdmap e562: 8 total, 8 up, 8 in 2024-02-05T08:02:57.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:57 smithi157 ceph-mon[33345]: osdmap e563: 8 total, 8 up, 8 in 2024-02-05T08:02:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:57 smithi131 ceph-mon[29507]: osdmap e562: 8 total, 8 up, 8 in 2024-02-05T08:02:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:57 smithi131 ceph-mon[29507]: osdmap e563: 8 total, 8 up, 8 in 2024-02-05T08:02:57.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:57 smithi131 ceph-mon[41654]: osdmap e562: 8 total, 8 up, 8 in 2024-02-05T08:02:57.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:57 smithi131 ceph-mon[41654]: osdmap e563: 8 total, 8 up, 8 in 2024-02-05T08:02:58.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:58 smithi157 ceph-mon[33345]: pgmap v868: 244 pgs: 244 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:58.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:58 smithi157 ceph-mon[33345]: osdmap e564: 8 total, 8 up, 8 in 2024-02-05T08:02:58.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:58 smithi131 ceph-mon[41654]: pgmap v868: 244 pgs: 244 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:58 smithi131 ceph-mon[41654]: osdmap e564: 8 total, 8 up, 8 in 2024-02-05T08:02:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:58.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:58 smithi131 ceph-mon[29507]: pgmap v868: 244 pgs: 244 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:02:58.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:58 smithi131 ceph-mon[29507]: osdmap e564: 8 total, 8 up, 8 in 2024-02-05T08:02:58.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:59.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:02:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:02:59.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:59 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:59.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-109"}]': finished 2024-02-05T08:02:59.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:59 smithi157 ceph-mon[33345]: osdmap e565: 8 total, 8 up, 8 in 2024-02-05T08:02:59.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:02:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:59.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:59 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-109"}]': finished 2024-02-05T08:02:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:59 smithi131 ceph-mon[29507]: osdmap e565: 8 total, 8 up, 8 in 2024-02-05T08:02:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:02:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:02:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:59 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-109"}]': finished 2024-02-05T08:02:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:59 smithi131 ceph-mon[41654]: osdmap e565: 8 total, 8 up, 8 in 2024-02-05T08:02:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:02:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-109"}]: dispatch 2024-02-05T08:03:00.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:00 smithi157 ceph-mon[33345]: pgmap v871: 236 pgs: 236 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:00.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:00.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-109"}]': finished 2024-02-05T08:03:00.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:00 smithi157 ceph-mon[33345]: osdmap e566: 8 total, 8 up, 8 in 2024-02-05T08:03:00.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:00 smithi131 ceph-mon[29507]: pgmap v871: 236 pgs: 236 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:00.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:00.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-109"}]': finished 2024-02-05T08:03:00.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:00 smithi131 ceph-mon[29507]: osdmap e566: 8 total, 8 up, 8 in 2024-02-05T08:03:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:00 smithi131 ceph-mon[41654]: pgmap v871: 236 pgs: 236 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3311611428' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-109"}]': finished 2024-02-05T08:03:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:00 smithi131 ceph-mon[41654]: osdmap e566: 8 total, 8 up, 8 in 2024-02-05T08:03:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:01 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:01 smithi157 ceph-mon[33345]: osdmap e567: 8 total, 8 up, 8 in 2024-02-05T08:03:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:03:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:03:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:01 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:01 smithi131 ceph-mon[41654]: osdmap e567: 8 total, 8 up, 8 in 2024-02-05T08:03:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:03:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:03:01.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:01 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:01.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:01 smithi131 ceph-mon[29507]: osdmap e567: 8 total, 8 up, 8 in 2024-02-05T08:03:01.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:03:01.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:03:02.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:02 smithi157 ceph-mon[33345]: pgmap v874: 228 pgs: 228 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T08:03:02.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104"}]': finished 2024-02-05T08:03:02.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:02 smithi157 ceph-mon[33345]: osdmap e568: 8 total, 8 up, 8 in 2024-02-05T08:03:02.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:03:02.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:03:03.017 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:03:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:03:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:03:03.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:02 smithi131 ceph-mon[41654]: pgmap v874: 228 pgs: 228 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T08:03:03.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104"}]': finished 2024-02-05T08:03:03.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:02 smithi131 ceph-mon[41654]: osdmap e568: 8 total, 8 up, 8 in 2024-02-05T08:03:03.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:03:03.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:03:03.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:02 smithi131 ceph-mon[29507]: pgmap v874: 228 pgs: 228 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T08:03:03.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-82290-104"}]': finished 2024-02-05T08:03:03.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:02 smithi131 ceph-mon[29507]: osdmap e568: 8 total, 8 up, 8 in 2024-02-05T08:03:03.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2125724941' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:03:03.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-82290-104"}]: dispatch 2024-02-05T08:03:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-82290-104"}]': finished 2024-02-05T08:03:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:04 smithi157 ceph-mon[33345]: osdmap e569: 8 total, 8 up, 8 in 2024-02-05T08:03:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T08:03:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T08:03:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:04 smithi157 ceph-mon[33345]: pgmap v877: 228 pgs: 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T08:03:04.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-82290-104"}]': finished 2024-02-05T08:03:04.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[41654]: osdmap e569: 8 total, 8 up, 8 in 2024-02-05T08:03:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T08:03:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T08:03:04.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[41654]: pgmap v877: 228 pgs: 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T08:03:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-82290-104"}]': finished 2024-02-05T08:03:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[29507]: osdmap e569: 8 total, 8 up, 8 in 2024-02-05T08:03:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T08:03:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T08:03:04.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:04 smithi131 ceph-mon[29507]: pgmap v877: 228 pgs: 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T08:03:05.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T08:03:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:05 smithi157 ceph-mon[33345]: osdmap e570: 8 total, 8 up, 8 in 2024-02-05T08:03:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:05.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:05.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T08:03:05.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:05 smithi131 ceph-mon[41654]: osdmap e570: 8 total, 8 up, 8 in 2024-02-05T08:03:05.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:05.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:05.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-82290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T08:03:05.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:05 smithi131 ceph-mon[29507]: osdmap e570: 8 total, 8 up, 8 in 2024-02-05T08:03:05.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:05.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:06.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:06 smithi157 ceph-mon[33345]: osdmap e571: 8 total, 8 up, 8 in 2024-02-05T08:03:06.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:06 smithi157 ceph-mon[33345]: pgmap v880: 228 pgs: 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:06.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:03:06.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:06 smithi157 ceph-mon[33345]: osdmap e572: 8 total, 8 up, 8 in 2024-02-05T08:03:06.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:06 smithi131 ceph-mon[41654]: osdmap e571: 8 total, 8 up, 8 in 2024-02-05T08:03:06.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:06 smithi131 ceph-mon[41654]: pgmap v880: 228 pgs: 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:06.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:03:06.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:06 smithi131 ceph-mon[41654]: osdmap e572: 8 total, 8 up, 8 in 2024-02-05T08:03:06.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:06 smithi131 ceph-mon[29507]: osdmap e571: 8 total, 8 up, 8 in 2024-02-05T08:03:06.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:06 smithi131 ceph-mon[29507]: pgmap v880: 228 pgs: 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:06.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-82290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:03:06.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:06 smithi131 ceph-mon[29507]: osdmap e572: 8 total, 8 up, 8 in 2024-02-05T08:03:07.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:07 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:07.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:07 smithi157 ceph-mon[33345]: osdmap e573: 8 total, 8 up, 8 in 2024-02-05T08:03:07.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:07 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:07.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:07.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:07 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:07.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:07 smithi131 ceph-mon[41654]: osdmap e573: 8 total, 8 up, 8 in 2024-02-05T08:03:07.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:07 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:07.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:07.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:07 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:07.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:07 smithi131 ceph-mon[29507]: osdmap e573: 8 total, 8 up, 8 in 2024-02-05T08:03:07.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:07 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:07.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:08 smithi157 ceph-mon[33345]: pgmap v883: 268 pgs: 40 unknown, 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:03:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:08 smithi157 ceph-mon[33345]: osdmap e574: 8 total, 8 up, 8 in 2024-02-05T08:03:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:08.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:08.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:08 smithi131 ceph-mon[41654]: pgmap v883: 268 pgs: 40 unknown, 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:03:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:08 smithi131 ceph-mon[41654]: osdmap e574: 8 total, 8 up, 8 in 2024-02-05T08:03:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:08.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:08.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:08 smithi131 ceph-mon[29507]: pgmap v883: 268 pgs: 40 unknown, 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:08.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:03:08.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:08 smithi131 ceph-mon[29507]: osdmap e574: 8 total, 8 up, 8 in 2024-02-05T08:03:08.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:08.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:09.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:03:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:03:09.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:09 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:09.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:03:09.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:09 smithi157 ceph-mon[33345]: osdmap e575: 8 total, 8 up, 8 in 2024-02-05T08:03:09.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-112"}]: dispatch 2024-02-05T08:03:09.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-112"}]: dispatch 2024-02-05T08:03:09.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:09 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:09.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:03:09.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:09 smithi131 ceph-mon[41654]: osdmap e575: 8 total, 8 up, 8 in 2024-02-05T08:03:09.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-112"}]: dispatch 2024-02-05T08:03:09.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-112"}]: dispatch 2024-02-05T08:03:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:09 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:03:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:09 smithi131 ceph-mon[29507]: osdmap e575: 8 total, 8 up, 8 in 2024-02-05T08:03:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-112"}]: dispatch 2024-02-05T08:03:09.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-112"}]: dispatch 2024-02-05T08:03:10.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:10 smithi157 ceph-mon[33345]: pgmap v886: 268 pgs: 268 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T08:03:10.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-112"}]': finished 2024-02-05T08:03:10.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:10 smithi157 ceph-mon[33345]: osdmap e576: 8 total, 8 up, 8 in 2024-02-05T08:03:10.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:10.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:10.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:10 smithi131 ceph-mon[41654]: pgmap v886: 268 pgs: 268 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T08:03:10.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-112"}]': finished 2024-02-05T08:03:10.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:10 smithi131 ceph-mon[41654]: osdmap e576: 8 total, 8 up, 8 in 2024-02-05T08:03:10.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:10.768 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:10 smithi131 ceph-mon[29507]: pgmap v886: 268 pgs: 268 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T08:03:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-112"}]': finished 2024-02-05T08:03:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:10 smithi131 ceph-mon[29507]: osdmap e576: 8 total, 8 up, 8 in 2024-02-05T08:03:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:10.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:12.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:03:12.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:12 smithi157 ceph-mon[33345]: osdmap e577: 8 total, 8 up, 8 in 2024-02-05T08:03:12.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:12 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112"}]: dispatch 2024-02-05T08:03:12.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:12 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112"}]: dispatch 2024-02-05T08:03:12.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:12 smithi157 ceph-mon[33345]: pgmap v889: 268 pgs: 268 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T08:03:12.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:12 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:12.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:03:12.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:12 smithi131 ceph-mon[41654]: osdmap e577: 8 total, 8 up, 8 in 2024-02-05T08:03:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:12 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112"}]: dispatch 2024-02-05T08:03:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:12 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112"}]: dispatch 2024-02-05T08:03:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:12 smithi131 ceph-mon[41654]: pgmap v889: 268 pgs: 268 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T08:03:12.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:12 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:12.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:03:12.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:12 smithi131 ceph-mon[29507]: osdmap e577: 8 total, 8 up, 8 in 2024-02-05T08:03:12.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:12 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112"}]: dispatch 2024-02-05T08:03:12.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:12 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112"}]: dispatch 2024-02-05T08:03:12.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:12 smithi131 ceph-mon[29507]: pgmap v889: 268 pgs: 268 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T08:03:12.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:12 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:13 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112"}]': finished 2024-02-05T08:03:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:13 smithi131 ceph-mon[41654]: osdmap e578: 8 total, 8 up, 8 in 2024-02-05T08:03:13.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:03:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:03:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:03:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:13 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112"}]': finished 2024-02-05T08:03:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:13 smithi131 ceph-mon[29507]: osdmap e578: 8 total, 8 up, 8 in 2024-02-05T08:03:13.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:13 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-112"}]': finished 2024-02-05T08:03:13.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:13 smithi157 ceph-mon[33345]: osdmap e578: 8 total, 8 up, 8 in 2024-02-05T08:03:14.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:14 smithi157 ceph-mon[33345]: osdmap e579: 8 total, 8 up, 8 in 2024-02-05T08:03:14.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:14 smithi157 ceph-mon[33345]: pgmap v892: 236 pgs: 236 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-05T08:03:14.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:14 smithi131 ceph-mon[41654]: osdmap e579: 8 total, 8 up, 8 in 2024-02-05T08:03:14.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:14 smithi131 ceph-mon[41654]: pgmap v892: 236 pgs: 236 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-05T08:03:14.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:14 smithi131 ceph-mon[29507]: osdmap e579: 8 total, 8 up, 8 in 2024-02-05T08:03:14.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:14 smithi131 ceph-mon[29507]: pgmap v892: 236 pgs: 236 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-05T08:03:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:15 smithi157 ceph-mon[33345]: osdmap e580: 8 total, 8 up, 8 in 2024-02-05T08:03:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:15.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:15.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:15 smithi131 ceph-mon[41654]: osdmap e580: 8 total, 8 up, 8 in 2024-02-05T08:03:15.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:15.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:15.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:15.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:15 smithi131 ceph-mon[29507]: osdmap e580: 8 total, 8 up, 8 in 2024-02-05T08:03:15.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:15.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:15.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:16.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:03:16.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:16 smithi157 ceph-mon[33345]: osdmap e581: 8 total, 8 up, 8 in 2024-02-05T08:03:16.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:16.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:16.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:16 smithi157 ceph-mon[33345]: pgmap v895: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-05T08:03:16.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:03:16.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:16 smithi131 ceph-mon[29507]: osdmap e581: 8 total, 8 up, 8 in 2024-02-05T08:03:16.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:16.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:16.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:16 smithi131 ceph-mon[29507]: pgmap v895: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-05T08:03:16.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:03:16.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:16 smithi131 ceph-mon[41654]: osdmap e581: 8 total, 8 up, 8 in 2024-02-05T08:03:16.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:16.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:16.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:16 smithi131 ceph-mon[41654]: pgmap v895: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-05T08:03:17.264 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:03:17.264 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:17 smithi131 ceph-mon[41654]: osdmap e582: 8 total, 8 up, 8 in 2024-02-05T08:03:17.264 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-114"}]: dispatch 2024-02-05T08:03:17.264 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-114"}]: dispatch 2024-02-05T08:03:17.264 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:17 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:03:17.264 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:03:17.264 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:17 smithi131 ceph-mon[29507]: osdmap e582: 8 total, 8 up, 8 in 2024-02-05T08:03:17.264 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-114"}]: dispatch 2024-02-05T08:03:17.265 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-114"}]: dispatch 2024-02-05T08:03:17.265 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:17 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:03:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:03:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:17 smithi157 ceph-mon[33345]: osdmap e582: 8 total, 8 up, 8 in 2024-02-05T08:03:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-114"}]: dispatch 2024-02-05T08:03:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-114"}]: dispatch 2024-02-05T08:03:17.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:17 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:03:18.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-114"}]': finished 2024-02-05T08:03:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:18 smithi157 ceph-mon[33345]: osdmap e583: 8 total, 8 up, 8 in 2024-02-05T08:03:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-114", "mode": "writeback"}]: dispatch 2024-02-05T08:03:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-114", "mode": "writeback"}]: dispatch 2024-02-05T08:03:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:18 smithi157 ceph-mon[33345]: pgmap v898: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:18 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:03:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-114", "mode": "writeback"}]': finished 2024-02-05T08:03:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:18 smithi157 ceph-mon[33345]: osdmap e584: 8 total, 8 up, 8 in 2024-02-05T08:03:18.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-114"}]': finished 2024-02-05T08:03:18.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:18 smithi131 ceph-mon[29507]: osdmap e583: 8 total, 8 up, 8 in 2024-02-05T08:03:18.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-114", "mode": "writeback"}]: dispatch 2024-02-05T08:03:18.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-114", "mode": "writeback"}]: dispatch 2024-02-05T08:03:18.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:18 smithi131 ceph-mon[29507]: pgmap v898: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:18.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:18 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:03:18.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-114", "mode": "writeback"}]': finished 2024-02-05T08:03:18.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:18 smithi131 ceph-mon[29507]: osdmap e584: 8 total, 8 up, 8 in 2024-02-05T08:03:18.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-114"}]': finished 2024-02-05T08:03:18.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:18 smithi131 ceph-mon[41654]: osdmap e583: 8 total, 8 up, 8 in 2024-02-05T08:03:18.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-114", "mode": "writeback"}]: dispatch 2024-02-05T08:03:18.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-114", "mode": "writeback"}]: dispatch 2024-02-05T08:03:18.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:18 smithi131 ceph-mon[41654]: pgmap v898: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:18.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:18 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:03:18.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-114", "mode": "writeback"}]': finished 2024-02-05T08:03:18.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:18 smithi131 ceph-mon[41654]: osdmap e584: 8 total, 8 up, 8 in 2024-02-05T08:03:19.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:03:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:03:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:19 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:19.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:19.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:19.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:19 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:19.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:19.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:19.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:19 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:20.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:20 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:03:20.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:20 smithi157 ceph-mon[33345]: osdmap e585: 8 total, 8 up, 8 in 2024-02-05T08:03:20.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:20 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114"}]: dispatch 2024-02-05T08:03:20.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:20 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114"}]: dispatch 2024-02-05T08:03:20.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:20 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:20.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:20 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:20.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:20 smithi157 ceph-mon[33345]: pgmap v901: 268 pgs: 268 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T08:03:20.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:20 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:03:20.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:20 smithi131 ceph-mon[41654]: osdmap e585: 8 total, 8 up, 8 in 2024-02-05T08:03:20.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:20 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114"}]: dispatch 2024-02-05T08:03:20.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:20 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114"}]: dispatch 2024-02-05T08:03:20.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:20 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:20.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:20 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:20.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:20 smithi131 ceph-mon[41654]: pgmap v901: 268 pgs: 268 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T08:03:20.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:20 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:03:20.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:20 smithi131 ceph-mon[29507]: osdmap e585: 8 total, 8 up, 8 in 2024-02-05T08:03:20.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:20 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114"}]: dispatch 2024-02-05T08:03:20.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:20 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114"}]: dispatch 2024-02-05T08:03:20.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:20 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:20.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:20 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:20.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:20 smithi131 ceph-mon[29507]: pgmap v901: 268 pgs: 268 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T08:03:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:21 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:03:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114"}]': finished 2024-02-05T08:03:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:21 smithi157 ceph-mon[33345]: osdmap e586: 8 total, 8 up, 8 in 2024-02-05T08:03:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:21 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:21 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:21 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:03:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:21 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:03:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:21 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:21 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:03:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114"}]': finished 2024-02-05T08:03:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:21 smithi131 ceph-mon[29507]: osdmap e586: 8 total, 8 up, 8 in 2024-02-05T08:03:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:21 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:21 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:21.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:21 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:03:21.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:21 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:03:21.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:21 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:21.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:21 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:03:21.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-114"}]': finished 2024-02-05T08:03:21.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:21 smithi131 ceph-mon[41654]: osdmap e586: 8 total, 8 up, 8 in 2024-02-05T08:03:21.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:21 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:21.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:21 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:21.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:21 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:03:21.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:21 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:03:21.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:21 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:22.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:22 smithi157 ceph-mon[33345]: osdmap e587: 8 total, 8 up, 8 in 2024-02-05T08:03:22.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:22 smithi157 ceph-mon[33345]: pgmap v904: 236 pgs: 236 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:03:22.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:22 smithi131 ceph-mon[41654]: osdmap e587: 8 total, 8 up, 8 in 2024-02-05T08:03:22.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:22 smithi131 ceph-mon[41654]: pgmap v904: 236 pgs: 236 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:03:22.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:22 smithi131 ceph-mon[29507]: osdmap e587: 8 total, 8 up, 8 in 2024-02-05T08:03:22.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:22 smithi131 ceph-mon[29507]: pgmap v904: 236 pgs: 236 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:03:23.232 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:03:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:03:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:03:23.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:23 smithi131 ceph-mon[41654]: osdmap e588: 8 total, 8 up, 8 in 2024-02-05T08:03:23.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:23.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:23.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:23 smithi131 ceph-mon[29507]: osdmap e588: 8 total, 8 up, 8 in 2024-02-05T08:03:23.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:23.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:23.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:23 smithi157 ceph-mon[33345]: osdmap e588: 8 total, 8 up, 8 in 2024-02-05T08:03:23.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:23.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:24.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:03:24.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:24 smithi131 ceph-mon[41654]: osdmap e589: 8 total, 8 up, 8 in 2024-02-05T08:03:24.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:24 smithi131 ceph-mon[41654]: pgmap v907: 268 pgs: 268 active+clean; 582 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-02-05T08:03:24.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:24 smithi131 ceph-mon[41654]: osdmap e590: 8 total, 8 up, 8 in 2024-02-05T08:03:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:03:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:24 smithi131 ceph-mon[29507]: osdmap e589: 8 total, 8 up, 8 in 2024-02-05T08:03:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:24 smithi131 ceph-mon[29507]: pgmap v907: 268 pgs: 268 active+clean; 582 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-02-05T08:03:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:24 smithi131 ceph-mon[29507]: osdmap e590: 8 total, 8 up, 8 in 2024-02-05T08:03:24.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:03:24.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:24 smithi157 ceph-mon[33345]: osdmap e589: 8 total, 8 up, 8 in 2024-02-05T08:03:24.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:24 smithi157 ceph-mon[33345]: pgmap v907: 268 pgs: 268 active+clean; 582 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-02-05T08:03:24.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:24 smithi157 ceph-mon[33345]: osdmap e590: 8 total, 8 up, 8 in 2024-02-05T08:03:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:25.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:25.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:25.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:25.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:25.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:26.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:03:26.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:26 smithi131 ceph-mon[41654]: osdmap e591: 8 total, 8 up, 8 in 2024-02-05T08:03:26.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-116"}]: dispatch 2024-02-05T08:03:26.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-116"}]: dispatch 2024-02-05T08:03:26.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:26 smithi131 ceph-mon[41654]: pgmap v910: 268 pgs: 268 active+clean; 582 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T08:03:26.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:03:26.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:26 smithi131 ceph-mon[29507]: osdmap e591: 8 total, 8 up, 8 in 2024-02-05T08:03:26.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-116"}]: dispatch 2024-02-05T08:03:26.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-116"}]: dispatch 2024-02-05T08:03:26.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:26 smithi131 ceph-mon[29507]: pgmap v910: 268 pgs: 268 active+clean; 582 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T08:03:26.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:03:26.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:26 smithi157 ceph-mon[33345]: osdmap e591: 8 total, 8 up, 8 in 2024-02-05T08:03:26.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-116"}]: dispatch 2024-02-05T08:03:26.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-116"}]: dispatch 2024-02-05T08:03:26.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:26 smithi157 ceph-mon[33345]: pgmap v910: 268 pgs: 268 active+clean; 582 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T08:03:27.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-116"}]': finished 2024-02-05T08:03:27.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:27 smithi131 ceph-mon[41654]: osdmap e592: 8 total, 8 up, 8 in 2024-02-05T08:03:27.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:27 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-116", "mode": "writeback"}]: dispatch 2024-02-05T08:03:27.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-116", "mode": "writeback"}]: dispatch 2024-02-05T08:03:27.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-116"}]': finished 2024-02-05T08:03:27.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:27 smithi131 ceph-mon[29507]: osdmap e592: 8 total, 8 up, 8 in 2024-02-05T08:03:27.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:27 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-116", "mode": "writeback"}]: dispatch 2024-02-05T08:03:27.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-116", "mode": "writeback"}]: dispatch 2024-02-05T08:03:27.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-116"}]': finished 2024-02-05T08:03:27.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:27 smithi157 ceph-mon[33345]: osdmap e592: 8 total, 8 up, 8 in 2024-02-05T08:03:27.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:27 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-116", "mode": "writeback"}]: dispatch 2024-02-05T08:03:27.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-116", "mode": "writeback"}]: dispatch 2024-02-05T08:03:28.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:28 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:03:28.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-116", "mode": "writeback"}]': finished 2024-02-05T08:03:28.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:28 smithi131 ceph-mon[41654]: osdmap e593: 8 total, 8 up, 8 in 2024-02-05T08:03:28.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "325.3"}]: dispatch 2024-02-05T08:03:28.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:28 smithi131 ceph-mon[41654]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.3"}]: dispatch 2024-02-05T08:03:28.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:28 smithi131 ceph-mon[41654]: pgmap v913: 268 pgs: 268 active+clean; 582 KiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:28.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:28 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:03:28.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-116", "mode": "writeback"}]': finished 2024-02-05T08:03:28.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:28 smithi131 ceph-mon[29507]: osdmap e593: 8 total, 8 up, 8 in 2024-02-05T08:03:28.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "325.3"}]: dispatch 2024-02-05T08:03:28.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:28 smithi131 ceph-mon[29507]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.3"}]: dispatch 2024-02-05T08:03:28.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:28 smithi131 ceph-mon[29507]: pgmap v913: 268 pgs: 268 active+clean; 582 KiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:28.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:28 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:03:28.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-116", "mode": "writeback"}]': finished 2024-02-05T08:03:28.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:28 smithi157 ceph-mon[33345]: osdmap e593: 8 total, 8 up, 8 in 2024-02-05T08:03:28.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "325.3"}]: dispatch 2024-02-05T08:03:28.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:28 smithi157 ceph-mon[33345]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.3"}]: dispatch 2024-02-05T08:03:28.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:28 smithi157 ceph-mon[33345]: pgmap v913: 268 pgs: 268 active+clean; 582 KiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:29.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:03:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:03:29.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:29 smithi157 ceph-mon[33345]: 325.3 scrub starts 2024-02-05T08:03:29.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:29 smithi157 ceph-mon[33345]: 325.3 scrub ok 2024-02-05T08:03:29.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:29 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:29.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:29 smithi131 ceph-mon[41654]: 325.3 scrub starts 2024-02-05T08:03:29.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:29 smithi131 ceph-mon[41654]: 325.3 scrub ok 2024-02-05T08:03:29.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:29 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:29.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:29 smithi131 ceph-mon[29507]: 325.3 scrub starts 2024-02-05T08:03:29.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:29 smithi131 ceph-mon[29507]: 325.3 scrub ok 2024-02-05T08:03:29.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:29 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:30 smithi157 ceph-mon[33345]: pgmap v914: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 578 B/s wr, 2 op/s 2024-02-05T08:03:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:30 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:30 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:30 smithi131 ceph-mon[41654]: pgmap v914: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 578 B/s wr, 2 op/s 2024-02-05T08:03:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:30 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:30 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:30 smithi131 ceph-mon[29507]: pgmap v914: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 578 B/s wr, 2 op/s 2024-02-05T08:03:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:30 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:03:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:30 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:33.119 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:32 smithi131 ceph-mon[41654]: pgmap v915: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 488 B/s wr, 2 op/s 2024-02-05T08:03:33.120 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:03:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:03:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:03:33.120 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:32 smithi131 ceph-mon[29507]: pgmap v915: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 488 B/s wr, 2 op/s 2024-02-05T08:03:33.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:32 smithi157 ceph-mon[33345]: pgmap v915: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 488 B/s wr, 2 op/s 2024-02-05T08:03:35.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:34 smithi157 ceph-mon[33345]: pgmap v916: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-02-05T08:03:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:34 smithi131 ceph-mon[41654]: pgmap v916: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-02-05T08:03:35.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:34 smithi131 ceph-mon[29507]: pgmap v916: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-02-05T08:03:37.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:36 smithi157 ceph-mon[33345]: pgmap v917: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 330 B/s wr, 1 op/s 2024-02-05T08:03:37.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:36 smithi131 ceph-mon[41654]: pgmap v917: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 330 B/s wr, 1 op/s 2024-02-05T08:03:37.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:36 smithi131 ceph-mon[29507]: pgmap v917: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 330 B/s wr, 1 op/s 2024-02-05T08:03:37.743 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.743 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.743 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.743 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.743 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.743 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.743 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.743 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.744 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.745 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.746 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.747 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.748 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.749 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.749 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.749 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.749 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.749 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.749 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.749 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.749 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.749 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.749 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.749 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.749 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.749 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.749 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.750 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.750 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.750 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.750 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.750 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.750 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:37.750 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:03:39.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:38 smithi157 ceph-mon[33345]: pgmap v918: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 298 B/s wr, 1 op/s 2024-02-05T08:03:39.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:03:38 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:03:39.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:38 smithi131 ceph-mon[41654]: pgmap v918: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 298 B/s wr, 1 op/s 2024-02-05T08:03:39.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:38 smithi131 ceph-mon[29507]: pgmap v918: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 298 B/s wr, 1 op/s 2024-02-05T08:03:40.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:39 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:40.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:39 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:40.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:39 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:40 smithi157 ceph-mon[33345]: pgmap v919: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:03:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:40 smithi131 ceph-mon[41654]: pgmap v919: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:03:41.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:40 smithi131 ceph-mon[29507]: pgmap v919: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:03:43.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:42 smithi157 ceph-mon[33345]: pgmap v920: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:42 smithi131 ceph-mon[41654]: pgmap v920: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:03:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:03:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:03:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:42 smithi131 ceph-mon[29507]: pgmap v920: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:44.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:43 smithi157 ceph-mon[33345]: osdmap e594: 8 total, 8 up, 8 in 2024-02-05T08:03:44.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:44.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:44.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:43 smithi131 ceph-mon[41654]: osdmap e594: 8 total, 8 up, 8 in 2024-02-05T08:03:44.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:44.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:44.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:43 smithi131 ceph-mon[29507]: osdmap e594: 8 total, 8 up, 8 in 2024-02-05T08:03:44.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:44.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:44 smithi157 ceph-mon[33345]: pgmap v922: 268 pgs: 268 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:03:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:03:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:44 smithi157 ceph-mon[33345]: osdmap e595: 8 total, 8 up, 8 in 2024-02-05T08:03:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116"}]: dispatch 2024-02-05T08:03:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116"}]: dispatch 2024-02-05T08:03:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:44 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:45.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:44 smithi131 ceph-mon[41654]: pgmap v922: 268 pgs: 268 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:03:45.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:03:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:44 smithi131 ceph-mon[41654]: osdmap e595: 8 total, 8 up, 8 in 2024-02-05T08:03:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116"}]: dispatch 2024-02-05T08:03:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116"}]: dispatch 2024-02-05T08:03:45.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:44 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:45.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:44 smithi131 ceph-mon[29507]: pgmap v922: 268 pgs: 268 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:03:45.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:03:45.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:44 smithi131 ceph-mon[29507]: osdmap e595: 8 total, 8 up, 8 in 2024-02-05T08:03:45.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116"}]: dispatch 2024-02-05T08:03:45.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116"}]: dispatch 2024-02-05T08:03:45.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:44 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:46.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:45 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:03:46.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116"}]': finished 2024-02-05T08:03:46.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:45 smithi157 ceph-mon[33345]: osdmap e596: 8 total, 8 up, 8 in 2024-02-05T08:03:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:45 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:03:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116"}]': finished 2024-02-05T08:03:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:45 smithi131 ceph-mon[41654]: osdmap e596: 8 total, 8 up, 8 in 2024-02-05T08:03:46.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:45 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:03:46.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-116"}]': finished 2024-02-05T08:03:46.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:45 smithi131 ceph-mon[29507]: osdmap e596: 8 total, 8 up, 8 in 2024-02-05T08:03:47.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:46 smithi157 ceph-mon[33345]: pgmap v925: 268 pgs: 268 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T08:03:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:46 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:46 smithi157 ceph-mon[33345]: osdmap e597: 8 total, 8 up, 8 in 2024-02-05T08:03:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:46 smithi157 ceph-mon[33345]: osdmap e598: 8 total, 8 up, 8 in 2024-02-05T08:03:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:46 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:46 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:47.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:46 smithi131 ceph-mon[41654]: pgmap v925: 268 pgs: 268 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T08:03:47.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:46 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:47.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:46 smithi131 ceph-mon[41654]: osdmap e597: 8 total, 8 up, 8 in 2024-02-05T08:03:47.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:46 smithi131 ceph-mon[41654]: osdmap e598: 8 total, 8 up, 8 in 2024-02-05T08:03:47.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:46 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:47.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:46 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:47.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:46 smithi131 ceph-mon[29507]: pgmap v925: 268 pgs: 268 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T08:03:47.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:46 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:47.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:46 smithi131 ceph-mon[29507]: osdmap e597: 8 total, 8 up, 8 in 2024-02-05T08:03:47.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:46 smithi131 ceph-mon[29507]: osdmap e598: 8 total, 8 up, 8 in 2024-02-05T08:03:47.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:46 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:47.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:46 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:48.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:48 smithi157 ceph-mon[33345]: pgmap v928: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:48.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:03:48.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:48 smithi157 ceph-mon[33345]: osdmap e599: 8 total, 8 up, 8 in 2024-02-05T08:03:49.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:48 smithi131 ceph-mon[41654]: pgmap v928: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:49.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:03:49.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:48 smithi131 ceph-mon[41654]: osdmap e599: 8 total, 8 up, 8 in 2024-02-05T08:03:49.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:48 smithi131 ceph-mon[29507]: pgmap v928: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:49.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:03:49.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:48 smithi131 ceph-mon[29507]: osdmap e599: 8 total, 8 up, 8 in 2024-02-05T08:03:49.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:03:48 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:03:50.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:49 smithi131 ceph-mon[41654]: osdmap e600: 8 total, 8 up, 8 in 2024-02-05T08:03:50.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:50.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:49 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:50.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:49 smithi131 ceph-mon[29507]: osdmap e600: 8 total, 8 up, 8 in 2024-02-05T08:03:50.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:50.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:50.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:49 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:50.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:49 smithi157 ceph-mon[33345]: osdmap e600: 8 total, 8 up, 8 in 2024-02-05T08:03:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:49 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:51.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:50 smithi131 ceph-mon[41654]: pgmap v931: 268 pgs: 268 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:51.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:03:51.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:50 smithi131 ceph-mon[41654]: osdmap e601: 8 total, 8 up, 8 in 2024-02-05T08:03:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:50 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-118"}]: dispatch 2024-02-05T08:03:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-118"}]: dispatch 2024-02-05T08:03:51.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:50 smithi131 ceph-mon[29507]: pgmap v931: 268 pgs: 268 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:51.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:03:51.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:50 smithi131 ceph-mon[29507]: osdmap e601: 8 total, 8 up, 8 in 2024-02-05T08:03:51.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:50 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-118"}]: dispatch 2024-02-05T08:03:51.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-118"}]: dispatch 2024-02-05T08:03:51.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:50 smithi157 ceph-mon[33345]: pgmap v931: 268 pgs: 268 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:51.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:03:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:50 smithi157 ceph-mon[33345]: osdmap e601: 8 total, 8 up, 8 in 2024-02-05T08:03:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:50 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-118"}]: dispatch 2024-02-05T08:03:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-118"}]: dispatch 2024-02-05T08:03:52.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-118"}]': finished 2024-02-05T08:03:52.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:51 smithi131 ceph-mon[41654]: osdmap e602: 8 total, 8 up, 8 in 2024-02-05T08:03:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-118", "mode": "writeback"}]: dispatch 2024-02-05T08:03:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-118", "mode": "writeback"}]: dispatch 2024-02-05T08:03:52.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:51 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:52.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-118"}]': finished 2024-02-05T08:03:52.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:51 smithi131 ceph-mon[29507]: osdmap e602: 8 total, 8 up, 8 in 2024-02-05T08:03:52.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-118", "mode": "writeback"}]: dispatch 2024-02-05T08:03:52.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-118", "mode": "writeback"}]: dispatch 2024-02-05T08:03:52.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:51 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:52.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-118"}]': finished 2024-02-05T08:03:52.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:51 smithi157 ceph-mon[33345]: osdmap e602: 8 total, 8 up, 8 in 2024-02-05T08:03:52.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-118", "mode": "writeback"}]: dispatch 2024-02-05T08:03:52.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-118", "mode": "writeback"}]: dispatch 2024-02-05T08:03:52.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:51 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:53.017 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:03:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:03:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:03:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:52 smithi131 ceph-mon[29507]: pgmap v934: 268 pgs: 268 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:03:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:52 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:03:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-118", "mode": "writeback"}]': finished 2024-02-05T08:03:53.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:52 smithi131 ceph-mon[29507]: osdmap e603: 8 total, 8 up, 8 in 2024-02-05T08:03:53.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:52 smithi131 ceph-mon[41654]: pgmap v934: 268 pgs: 268 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:03:53.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:52 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:03:53.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-118", "mode": "writeback"}]': finished 2024-02-05T08:03:53.019 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:52 smithi131 ceph-mon[41654]: osdmap e603: 8 total, 8 up, 8 in 2024-02-05T08:03:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:52 smithi157 ceph-mon[33345]: pgmap v934: 268 pgs: 268 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:03:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:52 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:03:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-118", "mode": "writeback"}]': finished 2024-02-05T08:03:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:52 smithi157 ceph-mon[33345]: osdmap e603: 8 total, 8 up, 8 in 2024-02-05T08:03:54.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:53 smithi131 ceph-mon[41654]: osdmap e604: 8 total, 8 up, 8 in 2024-02-05T08:03:54.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:54.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:54.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:53 smithi131 ceph-mon[29507]: osdmap e604: 8 total, 8 up, 8 in 2024-02-05T08:03:54.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:54.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:54.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:53 smithi157 ceph-mon[33345]: osdmap e604: 8 total, 8 up, 8 in 2024-02-05T08:03:54.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:54.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:03:55.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:54 smithi157 ceph-mon[33345]: pgmap v937: 268 pgs: 268 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:03:55.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:03:55.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:54 smithi157 ceph-mon[33345]: osdmap e605: 8 total, 8 up, 8 in 2024-02-05T08:03:55.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118"}]: dispatch 2024-02-05T08:03:55.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118"}]: dispatch 2024-02-05T08:03:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:54 smithi131 ceph-mon[41654]: pgmap v937: 268 pgs: 268 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:03:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:03:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:54 smithi131 ceph-mon[41654]: osdmap e605: 8 total, 8 up, 8 in 2024-02-05T08:03:55.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118"}]: dispatch 2024-02-05T08:03:55.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118"}]: dispatch 2024-02-05T08:03:55.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:54 smithi131 ceph-mon[29507]: pgmap v937: 268 pgs: 268 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:03:55.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:03:55.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:54 smithi131 ceph-mon[29507]: osdmap e605: 8 total, 8 up, 8 in 2024-02-05T08:03:55.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118"}]: dispatch 2024-02-05T08:03:55.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118"}]: dispatch 2024-02-05T08:03:56.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:55 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:03:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118"}]': finished 2024-02-05T08:03:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:55 smithi157 ceph-mon[33345]: osdmap e606: 8 total, 8 up, 8 in 2024-02-05T08:03:56.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:55 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:03:56.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118"}]': finished 2024-02-05T08:03:56.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:55 smithi131 ceph-mon[41654]: osdmap e606: 8 total, 8 up, 8 in 2024-02-05T08:03:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:55 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:03:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-118"}]': finished 2024-02-05T08:03:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:55 smithi131 ceph-mon[29507]: osdmap e606: 8 total, 8 up, 8 in 2024-02-05T08:03:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:56 smithi157 ceph-mon[33345]: pgmap v940: 268 pgs: 268 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:03:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:56 smithi157 ceph-mon[33345]: osdmap e607: 8 total, 8 up, 8 in 2024-02-05T08:03:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:56 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:56 smithi157 ceph-mon[33345]: osdmap e608: 8 total, 8 up, 8 in 2024-02-05T08:03:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:56 smithi131 ceph-mon[41654]: pgmap v940: 268 pgs: 268 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:03:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:56 smithi131 ceph-mon[41654]: osdmap e607: 8 total, 8 up, 8 in 2024-02-05T08:03:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:56 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:56 smithi131 ceph-mon[41654]: osdmap e608: 8 total, 8 up, 8 in 2024-02-05T08:03:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:56 smithi131 ceph-mon[29507]: pgmap v940: 268 pgs: 268 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:03:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:56 smithi131 ceph-mon[29507]: osdmap e607: 8 total, 8 up, 8 in 2024-02-05T08:03:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:56 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:03:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:56 smithi131 ceph-mon[29507]: osdmap e608: 8 total, 8 up, 8 in 2024-02-05T08:03:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:03:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:58 smithi157 ceph-mon[33345]: pgmap v943: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:03:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:58 smithi157 ceph-mon[33345]: osdmap e609: 8 total, 8 up, 8 in 2024-02-05T08:03:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:58.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:59.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:58 smithi131 ceph-mon[41654]: pgmap v943: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:59.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:03:59.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:58 smithi131 ceph-mon[41654]: osdmap e609: 8 total, 8 up, 8 in 2024-02-05T08:03:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:59.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:59.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:58 smithi131 ceph-mon[29507]: pgmap v943: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:03:59.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:03:59.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:58 smithi131 ceph-mon[29507]: osdmap e609: 8 total, 8 up, 8 in 2024-02-05T08:03:59.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:59.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:03:59.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:03:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:04:00.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:59 smithi131 ceph-mon[41654]: osdmap e610: 8 total, 8 up, 8 in 2024-02-05T08:04:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-120"}]: dispatch 2024-02-05T08:04:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-120"}]: dispatch 2024-02-05T08:04:00.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:03:59 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:00.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:00.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:59 smithi131 ceph-mon[29507]: osdmap e610: 8 total, 8 up, 8 in 2024-02-05T08:04:00.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-120"}]: dispatch 2024-02-05T08:04:00.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-120"}]: dispatch 2024-02-05T08:04:00.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:03:59 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:59 smithi157 ceph-mon[33345]: osdmap e610: 8 total, 8 up, 8 in 2024-02-05T08:04:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-120"}]: dispatch 2024-02-05T08:04:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-120"}]: dispatch 2024-02-05T08:04:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:03:59 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:01.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:00 smithi131 ceph-mon[41654]: pgmap v946: 268 pgs: 268 active+clean; 582 KiB data, 685 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:01.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-120"}]': finished 2024-02-05T08:04:01.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:00 smithi131 ceph-mon[41654]: osdmap e611: 8 total, 8 up, 8 in 2024-02-05T08:04:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-120", "mode": "writeback"}]: dispatch 2024-02-05T08:04:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-120", "mode": "writeback"}]: dispatch 2024-02-05T08:04:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:00 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:04:01.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:01.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:00 smithi131 ceph-mon[29507]: pgmap v946: 268 pgs: 268 active+clean; 582 KiB data, 685 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:01.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-120"}]': finished 2024-02-05T08:04:01.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:00 smithi131 ceph-mon[29507]: osdmap e611: 8 total, 8 up, 8 in 2024-02-05T08:04:01.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-120", "mode": "writeback"}]: dispatch 2024-02-05T08:04:01.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-120", "mode": "writeback"}]: dispatch 2024-02-05T08:04:01.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:00 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:04:01.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:01.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:00 smithi157 ceph-mon[33345]: pgmap v946: 268 pgs: 268 active+clean; 582 KiB data, 685 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-120"}]': finished 2024-02-05T08:04:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:00 smithi157 ceph-mon[33345]: osdmap e611: 8 total, 8 up, 8 in 2024-02-05T08:04:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-120", "mode": "writeback"}]: dispatch 2024-02-05T08:04:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-120", "mode": "writeback"}]: dispatch 2024-02-05T08:04:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:00 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:04:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:02.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:01 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-120", "mode": "writeback"}]': finished 2024-02-05T08:04:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:01 smithi131 ceph-mon[41654]: osdmap e612: 8 total, 8 up, 8 in 2024-02-05T08:04:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:02.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:01 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:02.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:01 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:02.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-120", "mode": "writeback"}]': finished 2024-02-05T08:04:02.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:01 smithi131 ceph-mon[29507]: osdmap e612: 8 total, 8 up, 8 in 2024-02-05T08:04:02.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:02.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:02.019 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:01 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:02.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:01 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-120", "mode": "writeback"}]': finished 2024-02-05T08:04:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:01 smithi157 ceph-mon[33345]: osdmap e612: 8 total, 8 up, 8 in 2024-02-05T08:04:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:01 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:03.017 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:04:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:04:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:04:03.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:02 smithi131 ceph-mon[41654]: pgmap v949: 268 pgs: 268 active+clean; 582 KiB data, 685 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:03.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:03.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:02 smithi131 ceph-mon[41654]: osdmap e613: 8 total, 8 up, 8 in 2024-02-05T08:04:03.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120"}]: dispatch 2024-02-05T08:04:03.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120"}]: dispatch 2024-02-05T08:04:03.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:02 smithi131 ceph-mon[29507]: pgmap v949: 268 pgs: 268 active+clean; 582 KiB data, 685 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:03.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:03.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:02 smithi131 ceph-mon[29507]: osdmap e613: 8 total, 8 up, 8 in 2024-02-05T08:04:03.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120"}]: dispatch 2024-02-05T08:04:03.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120"}]: dispatch 2024-02-05T08:04:03.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:02 smithi157 ceph-mon[33345]: pgmap v949: 268 pgs: 268 active+clean; 582 KiB data, 685 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:02 smithi157 ceph-mon[33345]: osdmap e613: 8 total, 8 up, 8 in 2024-02-05T08:04:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120"}]: dispatch 2024-02-05T08:04:03.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120"}]: dispatch 2024-02-05T08:04:04.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:03 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:04.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120"}]': finished 2024-02-05T08:04:04.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:03 smithi131 ceph-mon[41654]: osdmap e614: 8 total, 8 up, 8 in 2024-02-05T08:04:04.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:03 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:04.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120"}]': finished 2024-02-05T08:04:04.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:03 smithi131 ceph-mon[29507]: osdmap e614: 8 total, 8 up, 8 in 2024-02-05T08:04:04.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:03 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:04.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-120"}]': finished 2024-02-05T08:04:04.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:03 smithi157 ceph-mon[33345]: osdmap e614: 8 total, 8 up, 8 in 2024-02-05T08:04:05.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:04 smithi131 ceph-mon[41654]: pgmap v952: 268 pgs: 268 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-05T08:04:05.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:04 smithi131 ceph-mon[41654]: osdmap e615: 8 total, 8 up, 8 in 2024-02-05T08:04:05.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:04 smithi131 ceph-mon[29507]: pgmap v952: 268 pgs: 268 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-05T08:04:05.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:04 smithi131 ceph-mon[29507]: osdmap e615: 8 total, 8 up, 8 in 2024-02-05T08:04:05.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:04 smithi157 ceph-mon[33345]: pgmap v952: 268 pgs: 268 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-05T08:04:05.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:04 smithi157 ceph-mon[33345]: osdmap e615: 8 total, 8 up, 8 in 2024-02-05T08:04:06.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:05 smithi157 ceph-mon[33345]: osdmap e616: 8 total, 8 up, 8 in 2024-02-05T08:04:06.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:06.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:06.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:05 smithi131 ceph-mon[29507]: osdmap e616: 8 total, 8 up, 8 in 2024-02-05T08:04:06.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:06.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:06.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:05 smithi131 ceph-mon[41654]: osdmap e616: 8 total, 8 up, 8 in 2024-02-05T08:04:06.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:06.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:06 smithi157 ceph-mon[33345]: pgmap v955: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:06 smithi157 ceph-mon[33345]: osdmap e617: 8 total, 8 up, 8 in 2024-02-05T08:04:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:07.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:06 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:07.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:06 smithi131 ceph-mon[29507]: pgmap v955: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:07.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:06 smithi131 ceph-mon[29507]: osdmap e617: 8 total, 8 up, 8 in 2024-02-05T08:04:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:06 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:06 smithi131 ceph-mon[41654]: pgmap v955: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:06 smithi131 ceph-mon[41654]: osdmap e617: 8 total, 8 up, 8 in 2024-02-05T08:04:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:06 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:08.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:08.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:07 smithi157 ceph-mon[33345]: osdmap e618: 8 total, 8 up, 8 in 2024-02-05T08:04:08.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:07 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-122"}]: dispatch 2024-02-05T08:04:08.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-122"}]: dispatch 2024-02-05T08:04:08.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-122"}]': finished 2024-02-05T08:04:08.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:07 smithi157 ceph-mon[33345]: osdmap e619: 8 total, 8 up, 8 in 2024-02-05T08:04:08.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:08.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:07 smithi131 ceph-mon[29507]: osdmap e618: 8 total, 8 up, 8 in 2024-02-05T08:04:08.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:07 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-122"}]: dispatch 2024-02-05T08:04:08.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-122"}]: dispatch 2024-02-05T08:04:08.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-122"}]': finished 2024-02-05T08:04:08.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:07 smithi131 ceph-mon[29507]: osdmap e619: 8 total, 8 up, 8 in 2024-02-05T08:04:08.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:08.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:07 smithi131 ceph-mon[41654]: osdmap e618: 8 total, 8 up, 8 in 2024-02-05T08:04:08.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:07 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-122"}]: dispatch 2024-02-05T08:04:08.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-122"}]: dispatch 2024-02-05T08:04:08.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-122"}]': finished 2024-02-05T08:04:08.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:07 smithi131 ceph-mon[41654]: osdmap e619: 8 total, 8 up, 8 in 2024-02-05T08:04:09.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:04:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:04:09.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:08 smithi157 ceph-mon[33345]: pgmap v958: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:09.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-122", "mode": "writeback"}]: dispatch 2024-02-05T08:04:09.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-122", "mode": "writeback"}]: dispatch 2024-02-05T08:04:09.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:08 smithi131 ceph-mon[29507]: pgmap v958: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:09.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-122", "mode": "writeback"}]: dispatch 2024-02-05T08:04:09.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-122", "mode": "writeback"}]: dispatch 2024-02-05T08:04:09.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:08 smithi131 ceph-mon[41654]: pgmap v958: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:09.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-122", "mode": "writeback"}]: dispatch 2024-02-05T08:04:09.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-122", "mode": "writeback"}]: dispatch 2024-02-05T08:04:10.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:09 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-122", "mode": "writeback"}]': finished 2024-02-05T08:04:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:09 smithi157 ceph-mon[33345]: osdmap e620: 8 total, 8 up, 8 in 2024-02-05T08:04:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:09 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:10.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:09 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:10.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-122", "mode": "writeback"}]': finished 2024-02-05T08:04:10.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:09 smithi131 ceph-mon[41654]: osdmap e620: 8 total, 8 up, 8 in 2024-02-05T08:04:10.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:10.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:10.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:09 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:09 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-122", "mode": "writeback"}]': finished 2024-02-05T08:04:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:09 smithi131 ceph-mon[29507]: osdmap e620: 8 total, 8 up, 8 in 2024-02-05T08:04:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:10.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:09 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:11.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:10 smithi157 ceph-mon[33345]: pgmap v961: 268 pgs: 268 active+clean; 582 KiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:10 smithi157 ceph-mon[33345]: osdmap e621: 8 total, 8 up, 8 in 2024-02-05T08:04:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122"}]: dispatch 2024-02-05T08:04:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122"}]: dispatch 2024-02-05T08:04:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:10 smithi131 ceph-mon[41654]: pgmap v961: 268 pgs: 268 active+clean; 582 KiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:10 smithi131 ceph-mon[41654]: osdmap e621: 8 total, 8 up, 8 in 2024-02-05T08:04:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122"}]: dispatch 2024-02-05T08:04:11.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122"}]: dispatch 2024-02-05T08:04:11.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:10 smithi131 ceph-mon[29507]: pgmap v961: 268 pgs: 268 active+clean; 582 KiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:11.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:11.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:10 smithi131 ceph-mon[29507]: osdmap e621: 8 total, 8 up, 8 in 2024-02-05T08:04:11.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122"}]: dispatch 2024-02-05T08:04:11.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122"}]: dispatch 2024-02-05T08:04:12.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:11 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122"}]': finished 2024-02-05T08:04:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:11 smithi157 ceph-mon[33345]: osdmap e622: 8 total, 8 up, 8 in 2024-02-05T08:04:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:11 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:12.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:11 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:12.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122"}]': finished 2024-02-05T08:04:12.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:11 smithi131 ceph-mon[29507]: osdmap e622: 8 total, 8 up, 8 in 2024-02-05T08:04:12.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:11 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:12.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:11 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:12.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-122"}]': finished 2024-02-05T08:04:12.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:11 smithi131 ceph-mon[41654]: osdmap e622: 8 total, 8 up, 8 in 2024-02-05T08:04:12.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:11 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:13.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:12 smithi157 ceph-mon[33345]: pgmap v964: 268 pgs: 268 active+clean; 582 KiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:12 smithi157 ceph-mon[33345]: osdmap e623: 8 total, 8 up, 8 in 2024-02-05T08:04:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:12 smithi131 ceph-mon[41654]: pgmap v964: 268 pgs: 268 active+clean; 582 KiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:12 smithi131 ceph-mon[41654]: osdmap e623: 8 total, 8 up, 8 in 2024-02-05T08:04:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:12 smithi131 ceph-mon[29507]: pgmap v964: 268 pgs: 268 active+clean; 582 KiB data, 670 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:12 smithi131 ceph-mon[29507]: osdmap e623: 8 total, 8 up, 8 in 2024-02-05T08:04:13.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:04:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:04:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:04:14.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:13 smithi157 ceph-mon[33345]: osdmap e624: 8 total, 8 up, 8 in 2024-02-05T08:04:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:13 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:13 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:14.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:13 smithi131 ceph-mon[41654]: osdmap e624: 8 total, 8 up, 8 in 2024-02-05T08:04:14.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:13 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:14.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:13 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:14.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:13 smithi131 ceph-mon[29507]: osdmap e624: 8 total, 8 up, 8 in 2024-02-05T08:04:14.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:13 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:14.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:13 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:15.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:14 smithi157 ceph-mon[33345]: pgmap v967: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:14 smithi157 ceph-mon[33345]: osdmap e625: 8 total, 8 up, 8 in 2024-02-05T08:04:15.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:14 smithi131 ceph-mon[41654]: pgmap v967: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:15.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:15.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:14 smithi131 ceph-mon[41654]: osdmap e625: 8 total, 8 up, 8 in 2024-02-05T08:04:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:14 smithi131 ceph-mon[29507]: pgmap v967: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:14 smithi131 ceph-mon[29507]: osdmap e625: 8 total, 8 up, 8 in 2024-02-05T08:04:16.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:15 smithi157 ceph-mon[33345]: osdmap e626: 8 total, 8 up, 8 in 2024-02-05T08:04:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:16.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:15 smithi131 ceph-mon[41654]: osdmap e626: 8 total, 8 up, 8 in 2024-02-05T08:04:16.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:16.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:16.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:16.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:15 smithi131 ceph-mon[29507]: osdmap e626: 8 total, 8 up, 8 in 2024-02-05T08:04:16.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:16.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:16.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:17.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:16 smithi157 ceph-mon[33345]: pgmap v970: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:17.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:16 smithi157 ceph-mon[33345]: osdmap e627: 8 total, 8 up, 8 in 2024-02-05T08:04:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-124"}]: dispatch 2024-02-05T08:04:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-124"}]: dispatch 2024-02-05T08:04:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:16 smithi131 ceph-mon[29507]: pgmap v970: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:16 smithi131 ceph-mon[29507]: osdmap e627: 8 total, 8 up, 8 in 2024-02-05T08:04:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-124"}]: dispatch 2024-02-05T08:04:17.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-124"}]: dispatch 2024-02-05T08:04:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:16 smithi131 ceph-mon[41654]: pgmap v970: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:16 smithi131 ceph-mon[41654]: osdmap e627: 8 total, 8 up, 8 in 2024-02-05T08:04:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-124"}]: dispatch 2024-02-05T08:04:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-124"}]: dispatch 2024-02-05T08:04:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-124"}]': finished 2024-02-05T08:04:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:17 smithi157 ceph-mon[33345]: osdmap e628: 8 total, 8 up, 8 in 2024-02-05T08:04:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-124", "mode": "writeback"}]: dispatch 2024-02-05T08:04:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-124", "mode": "writeback"}]: dispatch 2024-02-05T08:04:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:17 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-124", "mode": "writeback"}]': finished 2024-02-05T08:04:18.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:17 smithi157 ceph-mon[33345]: osdmap e629: 8 total, 8 up, 8 in 2024-02-05T08:04:18.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-124"}]': finished 2024-02-05T08:04:18.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:17 smithi131 ceph-mon[41654]: osdmap e628: 8 total, 8 up, 8 in 2024-02-05T08:04:18.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-124", "mode": "writeback"}]: dispatch 2024-02-05T08:04:18.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-124", "mode": "writeback"}]: dispatch 2024-02-05T08:04:18.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:17 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:18.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-124", "mode": "writeback"}]': finished 2024-02-05T08:04:18.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:17 smithi131 ceph-mon[41654]: osdmap e629: 8 total, 8 up, 8 in 2024-02-05T08:04:18.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-124"}]': finished 2024-02-05T08:04:18.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:17 smithi131 ceph-mon[29507]: osdmap e628: 8 total, 8 up, 8 in 2024-02-05T08:04:18.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-124", "mode": "writeback"}]: dispatch 2024-02-05T08:04:18.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-124", "mode": "writeback"}]: dispatch 2024-02-05T08:04:18.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:17 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:18.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-124", "mode": "writeback"}]': finished 2024-02-05T08:04:18.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:17 smithi131 ceph-mon[29507]: osdmap e629: 8 total, 8 up, 8 in 2024-02-05T08:04:19.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:18 smithi157 ceph-mon[33345]: pgmap v973: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 688 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:19.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:04:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:04:19.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:18 smithi131 ceph-mon[41654]: pgmap v973: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 688 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:19.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:18 smithi131 ceph-mon[29507]: pgmap v973: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 688 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:20.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:19 smithi157 ceph-mon[33345]: osdmap e630: 8 total, 8 up, 8 in 2024-02-05T08:04:20.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:20.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:19 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:20.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:19 smithi131 ceph-mon[41654]: osdmap e630: 8 total, 8 up, 8 in 2024-02-05T08:04:20.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:20.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:20.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:19 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:20.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:19 smithi131 ceph-mon[29507]: osdmap e630: 8 total, 8 up, 8 in 2024-02-05T08:04:20.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:20.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:20.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:19 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:20 smithi157 ceph-mon[33345]: pgmap v976: 268 pgs: 268 active+clean; 582 KiB data, 689 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-05T08:04:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:20 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:20 smithi157 ceph-mon[33345]: osdmap e631: 8 total, 8 up, 8 in 2024-02-05T08:04:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:20 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124"}]: dispatch 2024-02-05T08:04:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:20 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124"}]: dispatch 2024-02-05T08:04:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:20 smithi131 ceph-mon[29507]: pgmap v976: 268 pgs: 268 active+clean; 582 KiB data, 689 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-05T08:04:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:20 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:20 smithi131 ceph-mon[29507]: osdmap e631: 8 total, 8 up, 8 in 2024-02-05T08:04:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:20 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124"}]: dispatch 2024-02-05T08:04:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:20 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124"}]: dispatch 2024-02-05T08:04:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:20 smithi131 ceph-mon[41654]: pgmap v976: 268 pgs: 268 active+clean; 582 KiB data, 689 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-05T08:04:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:20 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:20 smithi131 ceph-mon[41654]: osdmap e631: 8 total, 8 up, 8 in 2024-02-05T08:04:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:20 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124"}]: dispatch 2024-02-05T08:04:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:20 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124"}]: dispatch 2024-02-05T08:04:22.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:21 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:22.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124"}]': finished 2024-02-05T08:04:22.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:21 smithi131 ceph-mon[41654]: osdmap e632: 8 total, 8 up, 8 in 2024-02-05T08:04:22.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:21 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:04:22.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:21 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:22.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124"}]': finished 2024-02-05T08:04:22.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:21 smithi131 ceph-mon[29507]: osdmap e632: 8 total, 8 up, 8 in 2024-02-05T08:04:22.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:21 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:04:22.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:21 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:22.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-124"}]': finished 2024-02-05T08:04:22.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:21 smithi157 ceph-mon[33345]: osdmap e632: 8 total, 8 up, 8 in 2024-02-05T08:04:22.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:21 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:04:23.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:22 smithi157 ceph-mon[33345]: pgmap v979: 268 pgs: 268 active+clean; 582 KiB data, 689 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-05T08:04:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:22 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:22 smithi157 ceph-mon[33345]: osdmap e633: 8 total, 8 up, 8 in 2024-02-05T08:04:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:22 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:04:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:22 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:04:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:22 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:04:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:04:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:04:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:04:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:22 smithi131 ceph-mon[41654]: pgmap v979: 268 pgs: 268 active+clean; 582 KiB data, 689 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-05T08:04:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:22 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:22 smithi131 ceph-mon[41654]: osdmap e633: 8 total, 8 up, 8 in 2024-02-05T08:04:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:22 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:04:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:22 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:04:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:22 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:04:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:22 smithi131 ceph-mon[29507]: pgmap v979: 268 pgs: 268 active+clean; 582 KiB data, 689 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-05T08:04:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:22 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:22 smithi131 ceph-mon[29507]: osdmap e633: 8 total, 8 up, 8 in 2024-02-05T08:04:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:22 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:04:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:22 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:04:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:22 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:04:24.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:23 smithi157 ceph-mon[33345]: osdmap e634: 8 total, 8 up, 8 in 2024-02-05T08:04:24.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:24.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:24.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:23 smithi131 ceph-mon[41654]: osdmap e634: 8 total, 8 up, 8 in 2024-02-05T08:04:24.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:24.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:23 smithi131 ceph-mon[29507]: osdmap e634: 8 total, 8 up, 8 in 2024-02-05T08:04:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:24 smithi157 ceph-mon[33345]: pgmap v982: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:24 smithi157 ceph-mon[33345]: osdmap e635: 8 total, 8 up, 8 in 2024-02-05T08:04:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:24 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:25.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:24 smithi131 ceph-mon[41654]: pgmap v982: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:25.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:25.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:24 smithi131 ceph-mon[41654]: osdmap e635: 8 total, 8 up, 8 in 2024-02-05T08:04:25.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:24 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:25.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:25.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:24 smithi131 ceph-mon[29507]: pgmap v982: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:25.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:25.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:24 smithi131 ceph-mon[29507]: osdmap e635: 8 total, 8 up, 8 in 2024-02-05T08:04:25.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:25.271 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:26.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:26.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:25 smithi157 ceph-mon[33345]: osdmap e636: 8 total, 8 up, 8 in 2024-02-05T08:04:26.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-126"}]: dispatch 2024-02-05T08:04:26.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-126"}]: dispatch 2024-02-05T08:04:26.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:26.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:25 smithi131 ceph-mon[29507]: osdmap e636: 8 total, 8 up, 8 in 2024-02-05T08:04:26.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-126"}]: dispatch 2024-02-05T08:04:26.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-126"}]: dispatch 2024-02-05T08:04:26.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:26.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:25 smithi131 ceph-mon[41654]: osdmap e636: 8 total, 8 up, 8 in 2024-02-05T08:04:26.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-126"}]: dispatch 2024-02-05T08:04:26.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-126"}]: dispatch 2024-02-05T08:04:27.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:26 smithi157 ceph-mon[33345]: pgmap v985: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-126"}]': finished 2024-02-05T08:04:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:26 smithi157 ceph-mon[33345]: osdmap e637: 8 total, 8 up, 8 in 2024-02-05T08:04:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-126", "mode": "writeback"}]: dispatch 2024-02-05T08:04:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-126", "mode": "writeback"}]: dispatch 2024-02-05T08:04:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:26 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-126", "mode": "writeback"}]': finished 2024-02-05T08:04:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:26 smithi157 ceph-mon[33345]: osdmap e638: 8 total, 8 up, 8 in 2024-02-05T08:04:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:27.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[29507]: pgmap v985: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:27.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-126"}]': finished 2024-02-05T08:04:27.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[29507]: osdmap e637: 8 total, 8 up, 8 in 2024-02-05T08:04:27.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-126", "mode": "writeback"}]: dispatch 2024-02-05T08:04:27.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-126", "mode": "writeback"}]: dispatch 2024-02-05T08:04:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-126", "mode": "writeback"}]': finished 2024-02-05T08:04:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[29507]: osdmap e638: 8 total, 8 up, 8 in 2024-02-05T08:04:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[41654]: pgmap v985: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-126"}]': finished 2024-02-05T08:04:27.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[41654]: osdmap e637: 8 total, 8 up, 8 in 2024-02-05T08:04:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-126", "mode": "writeback"}]: dispatch 2024-02-05T08:04:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-126", "mode": "writeback"}]: dispatch 2024-02-05T08:04:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-126", "mode": "writeback"}]': finished 2024-02-05T08:04:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[41654]: osdmap e638: 8 total, 8 up, 8 in 2024-02-05T08:04:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:27.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:29.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:28 smithi131 ceph-mon[29507]: pgmap v988: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:29.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:29.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:28 smithi131 ceph-mon[29507]: osdmap e639: 8 total, 8 up, 8 in 2024-02-05T08:04:29.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126"}]: dispatch 2024-02-05T08:04:29.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126"}]: dispatch 2024-02-05T08:04:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:28 smithi131 ceph-mon[41654]: pgmap v988: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:28 smithi131 ceph-mon[41654]: osdmap e639: 8 total, 8 up, 8 in 2024-02-05T08:04:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126"}]: dispatch 2024-02-05T08:04:29.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126"}]: dispatch 2024-02-05T08:04:29.018 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:28 smithi157 ceph-mon[33345]: pgmap v988: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:29.019 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:29.019 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:28 smithi157 ceph-mon[33345]: osdmap e639: 8 total, 8 up, 8 in 2024-02-05T08:04:29.019 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126"}]: dispatch 2024-02-05T08:04:29.019 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126"}]: dispatch 2024-02-05T08:04:29.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:04:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:04:29.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 se api_tier_pp: require_osd_release = squid 2024-02-05T08:04:29.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9067 ms) 2024-02-05T08:04:29.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-02-05T08:04:29.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:04:29.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6043 ms) 2024-02-05T08:04:29.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-02-05T08:04:29.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:04:29.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5088 ms) 2024-02-05T08:04:29.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-02-05T08:04:29.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:04:29.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25311 ms) 2024-02-05T08:04:29.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-02-05T08:04:29.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:04:29.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14057 ms) 2024-02-05T08:04:29.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-02-05T08:04:29.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12072 ms) 2024-02-05T08:04:29.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-02-05T08:04:29.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-02-05T08:04:29.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: 00000010 69 73 |is| 2024-02-05T08:04:29.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: 00000012 2024-02-05T08:04:29.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12063 ms) 2024-02-05T08:04:29.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-02-05T08:04:29.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T08:04:29.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6047 ms) 2024-02-05T08:04:29.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (558481 ms total) 2024-02-05T08:04:29.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: 2024-02-05T08:04:29.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-02-05T08:04:29.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-02-05T08:04:29.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1012 ms) 2024-02-05T08:04:29.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-02-05T08:04:29.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11028 ms) 2024-02-05T08:04:29.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-02-05T08:04:29.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18089 ms) 2024-02-05T08:04:29.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-02-05T08:04:29.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-02-05T08:04:29.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30132 ms total) 2024-02-05T08:04:29.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: 2024-02-05T08:04:29.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-02-05T08:04:29.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-02-05T08:04:29.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7016 ms) 2024-02-05T08:04:29.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-02-05T08:04:29.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8071 ms) 2024-02-05T08:04:29.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-02-05T08:04:29.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: waiting for scrub... 2024-02-05T08:04:29.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: done waiting 2024-02-05T08:04:29.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24701 ms) 2024-02-05T08:04:29.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-02-05T08:04:29.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9859 ms) 2024-02-05T08:04:29.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-02-05T08:04:29.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (7981 ms) 2024-02-05T08:04:29.760 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-02-05T08:04:29.760 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8050 ms) 2024-02-05T08:04:29.760 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-02-05T08:04:29.760 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10090 ms) 2024-02-05T08:04:29.760 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-02-05T08:04:30.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:29 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:30.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126"}]': finished 2024-02-05T08:04:30.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:29 smithi131 ceph-mon[41654]: osdmap e640: 8 total, 8 up, 8 in 2024-02-05T08:04:30.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:29 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:30.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:29 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:30.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126"}]': finished 2024-02-05T08:04:30.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:29 smithi131 ceph-mon[29507]: osdmap e640: 8 total, 8 up, 8 in 2024-02-05T08:04:30.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:29 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:29 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-126"}]': finished 2024-02-05T08:04:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:29 smithi157 ceph-mon[33345]: osdmap e640: 8 total, 8 up, 8 in 2024-02-05T08:04:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:29 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:31.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:30 smithi131 ceph-mon[41654]: pgmap v991: 268 pgs: 268 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T08:04:31.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:30 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:31.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:30 smithi131 ceph-mon[41654]: osdmap e641: 8 total, 8 up, 8 in 2024-02-05T08:04:31.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:30 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:31.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:30 smithi131 ceph-mon[29507]: pgmap v991: 268 pgs: 268 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T08:04:31.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:30 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:31.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:30 smithi131 ceph-mon[29507]: osdmap e641: 8 total, 8 up, 8 in 2024-02-05T08:04:31.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:30 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:30 smithi157 ceph-mon[33345]: pgmap v991: 268 pgs: 268 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T08:04:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:30 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:30 smithi157 ceph-mon[33345]: osdmap e641: 8 total, 8 up, 8 in 2024-02-05T08:04:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:30 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:32.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:31 smithi131 ceph-mon[41654]: osdmap e642: 8 total, 8 up, 8 in 2024-02-05T08:04:32.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:31 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:32.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:31 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:32.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:31 smithi131 ceph-mon[29507]: osdmap e642: 8 total, 8 up, 8 in 2024-02-05T08:04:32.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:31 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:32.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:31 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:32.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:31 smithi157 ceph-mon[33345]: osdmap e642: 8 total, 8 up, 8 in 2024-02-05T08:04:32.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:31 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:32.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:31 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:32 smithi157 ceph-mon[33345]: pgmap v994: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:32 smithi157 ceph-mon[33345]: osdmap e643: 8 total, 8 up, 8 in 2024-02-05T08:04:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:32 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:04:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:04:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:04:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:32 smithi131 ceph-mon[41654]: pgmap v994: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:32 smithi131 ceph-mon[41654]: osdmap e643: 8 total, 8 up, 8 in 2024-02-05T08:04:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:32 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:32 smithi131 ceph-mon[29507]: pgmap v994: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:32 smithi131 ceph-mon[29507]: osdmap e643: 8 total, 8 up, 8 in 2024-02-05T08:04:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:32 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:34.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:34.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:33 smithi157 ceph-mon[33345]: osdmap e644: 8 total, 8 up, 8 in 2024-02-05T08:04:34.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:33 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-128"}]: dispatch 2024-02-05T08:04:34.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-128"}]: dispatch 2024-02-05T08:04:34.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:34.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:33 smithi131 ceph-mon[41654]: osdmap e644: 8 total, 8 up, 8 in 2024-02-05T08:04:34.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:33 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-128"}]: dispatch 2024-02-05T08:04:34.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-128"}]: dispatch 2024-02-05T08:04:34.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:34.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:33 smithi131 ceph-mon[29507]: osdmap e644: 8 total, 8 up, 8 in 2024-02-05T08:04:34.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:33 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-128"}]: dispatch 2024-02-05T08:04:34.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-128"}]: dispatch 2024-02-05T08:04:35.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:34 smithi157 ceph-mon[33345]: pgmap v997: 268 pgs: 268 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T08:04:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-128"}]': finished 2024-02-05T08:04:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:34 smithi157 ceph-mon[33345]: osdmap e645: 8 total, 8 up, 8 in 2024-02-05T08:04:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-128", "mode": "writeback"}]: dispatch 2024-02-05T08:04:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-128", "mode": "writeback"}]: dispatch 2024-02-05T08:04:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:34 smithi131 ceph-mon[41654]: pgmap v997: 268 pgs: 268 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T08:04:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-128"}]': finished 2024-02-05T08:04:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:34 smithi131 ceph-mon[41654]: osdmap e645: 8 total, 8 up, 8 in 2024-02-05T08:04:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-128", "mode": "writeback"}]: dispatch 2024-02-05T08:04:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-128", "mode": "writeback"}]: dispatch 2024-02-05T08:04:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:34 smithi131 ceph-mon[29507]: pgmap v997: 268 pgs: 268 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T08:04:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-128"}]': finished 2024-02-05T08:04:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:34 smithi131 ceph-mon[29507]: osdmap e645: 8 total, 8 up, 8 in 2024-02-05T08:04:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-128", "mode": "writeback"}]: dispatch 2024-02-05T08:04:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-128", "mode": "writeback"}]: dispatch 2024-02-05T08:04:36.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:35 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-128", "mode": "writeback"}]': finished 2024-02-05T08:04:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:35 smithi157 ceph-mon[33345]: osdmap e646: 8 total, 8 up, 8 in 2024-02-05T08:04:36.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:35 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:36.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-128", "mode": "writeback"}]': finished 2024-02-05T08:04:36.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:35 smithi131 ceph-mon[41654]: osdmap e646: 8 total, 8 up, 8 in 2024-02-05T08:04:36.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:35 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:36.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-128", "mode": "writeback"}]': finished 2024-02-05T08:04:36.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:35 smithi131 ceph-mon[29507]: osdmap e646: 8 total, 8 up, 8 in 2024-02-05T08:04:37.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:36 smithi157 ceph-mon[33345]: pgmap v1000: 268 pgs: 268 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:36 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:37.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:36 smithi131 ceph-mon[41654]: pgmap v1000: 268 pgs: 268 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:37.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:36 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:37.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:36 smithi131 ceph-mon[29507]: pgmap v1000: 268 pgs: 268 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:37.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:36 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:04:39.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:04:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:04:39.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:38 smithi157 ceph-mon[33345]: pgmap v1001: 268 pgs: 268 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 883 B/s rd, 0 op/s 2024-02-05T08:04:39.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:38 smithi131 ceph-mon[41654]: pgmap v1001: 268 pgs: 268 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 883 B/s rd, 0 op/s 2024-02-05T08:04:39.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:38 smithi131 ceph-mon[29507]: pgmap v1001: 268 pgs: 268 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 883 B/s rd, 0 op/s 2024-02-05T08:04:40.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:39 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:40.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:39 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:40.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:39 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:40 smithi157 ceph-mon[33345]: pgmap v1002: 268 pgs: 268 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 301 B/s wr, 1 op/s 2024-02-05T08:04:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:40 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:41.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:40 smithi131 ceph-mon[41654]: pgmap v1002: 268 pgs: 268 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 301 B/s wr, 1 op/s 2024-02-05T08:04:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:40 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:41.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:40 smithi131 ceph-mon[29507]: pgmap v1002: 268 pgs: 268 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 301 B/s wr, 1 op/s 2024-02-05T08:04:41.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:40 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:41.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:42.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:42.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:41 smithi157 ceph-mon[33345]: osdmap e647: 8 total, 8 up, 8 in 2024-02-05T08:04:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128"}]: dispatch 2024-02-05T08:04:42.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128"}]: dispatch 2024-02-05T08:04:42.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:42.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:41 smithi131 ceph-mon[41654]: osdmap e647: 8 total, 8 up, 8 in 2024-02-05T08:04:42.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128"}]: dispatch 2024-02-05T08:04:42.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128"}]: dispatch 2024-02-05T08:04:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:41 smithi131 ceph-mon[29507]: osdmap e647: 8 total, 8 up, 8 in 2024-02-05T08:04:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128"}]: dispatch 2024-02-05T08:04:42.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128"}]: dispatch 2024-02-05T08:04:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:04:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:04:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:04:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:43 smithi131 ceph-mon[41654]: pgmap v1004: 268 pgs: 268 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 789 B/s rd, 263 B/s wr, 1 op/s 2024-02-05T08:04:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:43 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128"}]': finished 2024-02-05T08:04:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:43 smithi131 ceph-mon[41654]: osdmap e648: 8 total, 8 up, 8 in 2024-02-05T08:04:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:43 smithi131 ceph-mon[29507]: pgmap v1004: 268 pgs: 268 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 789 B/s rd, 263 B/s wr, 1 op/s 2024-02-05T08:04:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:43 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128"}]': finished 2024-02-05T08:04:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:43 smithi131 ceph-mon[29507]: osdmap e648: 8 total, 8 up, 8 in 2024-02-05T08:04:43.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:43 smithi157 ceph-mon[33345]: pgmap v1004: 268 pgs: 268 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 789 B/s rd, 263 B/s wr, 1 op/s 2024-02-05T08:04:43.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:43 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:43.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-128"}]': finished 2024-02-05T08:04:43.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:43 smithi157 ceph-mon[33345]: osdmap e648: 8 total, 8 up, 8 in 2024-02-05T08:04:44.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:44 smithi157 ceph-mon[33345]: osdmap e649: 8 total, 8 up, 8 in 2024-02-05T08:04:44.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:44 smithi157 ceph-mon[33345]: osdmap e650: 8 total, 8 up, 8 in 2024-02-05T08:04:44.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:44 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:44.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:44 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:44.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:44 smithi131 ceph-mon[41654]: osdmap e649: 8 total, 8 up, 8 in 2024-02-05T08:04:44.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:44 smithi131 ceph-mon[41654]: osdmap e650: 8 total, 8 up, 8 in 2024-02-05T08:04:44.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:44 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:44.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:44 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:44.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:44 smithi131 ceph-mon[29507]: osdmap e649: 8 total, 8 up, 8 in 2024-02-05T08:04:44.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:44 smithi131 ceph-mon[29507]: osdmap e650: 8 total, 8 up, 8 in 2024-02-05T08:04:44.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:44 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:44.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:44 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:45.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:45 smithi157 ceph-mon[33345]: pgmap v1007: 236 pgs: 236 active+clean; 582 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T08:04:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:45 smithi157 ceph-mon[33345]: osdmap e651: 8 total, 8 up, 8 in 2024-02-05T08:04:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:45 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:45 smithi131 ceph-mon[41654]: pgmap v1007: 236 pgs: 236 active+clean; 582 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T08:04:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:45 smithi131 ceph-mon[41654]: osdmap e651: 8 total, 8 up, 8 in 2024-02-05T08:04:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:45 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:45.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:45 smithi131 ceph-mon[29507]: pgmap v1007: 236 pgs: 236 active+clean; 582 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T08:04:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:45 smithi131 ceph-mon[29507]: osdmap e651: 8 total, 8 up, 8 in 2024-02-05T08:04:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:45 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:47.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:46 smithi157 ceph-mon[33345]: pgmap v1010: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:04:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:46 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:46 smithi157 ceph-mon[33345]: osdmap e652: 8 total, 8 up, 8 in 2024-02-05T08:04:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:46 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-130"}]: dispatch 2024-02-05T08:04:47.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:46 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-130"}]: dispatch 2024-02-05T08:04:47.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:46 smithi131 ceph-mon[29507]: pgmap v1010: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:04:47.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:46 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:47.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:46 smithi131 ceph-mon[29507]: osdmap e652: 8 total, 8 up, 8 in 2024-02-05T08:04:47.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:46 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-130"}]: dispatch 2024-02-05T08:04:47.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:46 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-130"}]: dispatch 2024-02-05T08:04:47.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:46 smithi131 ceph-mon[41654]: pgmap v1010: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:04:47.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:46 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:47.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:46 smithi131 ceph-mon[41654]: osdmap e652: 8 total, 8 up, 8 in 2024-02-05T08:04:47.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:46 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-130"}]: dispatch 2024-02-05T08:04:47.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:46 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-130"}]: dispatch 2024-02-05T08:04:48.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-130"}]': finished 2024-02-05T08:04:48.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:47 smithi157 ceph-mon[33345]: osdmap e653: 8 total, 8 up, 8 in 2024-02-05T08:04:48.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:47 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-130", "mode": "writeback"}]: dispatch 2024-02-05T08:04:48.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-130", "mode": "writeback"}]: dispatch 2024-02-05T08:04:48.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-130"}]': finished 2024-02-05T08:04:48.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:47 smithi131 ceph-mon[29507]: osdmap e653: 8 total, 8 up, 8 in 2024-02-05T08:04:48.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:47 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-130", "mode": "writeback"}]: dispatch 2024-02-05T08:04:48.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-130", "mode": "writeback"}]: dispatch 2024-02-05T08:04:48.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-130"}]': finished 2024-02-05T08:04:48.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:47 smithi131 ceph-mon[41654]: osdmap e653: 8 total, 8 up, 8 in 2024-02-05T08:04:48.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:47 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-130", "mode": "writeback"}]: dispatch 2024-02-05T08:04:48.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-130", "mode": "writeback"}]: dispatch 2024-02-05T08:04:49.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:04:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:04:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:48 smithi157 ceph-mon[33345]: pgmap v1013: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 696 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:48 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-130", "mode": "writeback"}]': finished 2024-02-05T08:04:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:48 smithi157 ceph-mon[33345]: osdmap e654: 8 total, 8 up, 8 in 2024-02-05T08:04:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:49.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:48 smithi131 ceph-mon[29507]: pgmap v1013: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 696 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:49.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:48 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:49.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-130", "mode": "writeback"}]': finished 2024-02-05T08:04:49.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:48 smithi131 ceph-mon[29507]: osdmap e654: 8 total, 8 up, 8 in 2024-02-05T08:04:49.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:49.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:49.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:48 smithi131 ceph-mon[41654]: pgmap v1013: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 696 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:49.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:48 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:49.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-130", "mode": "writeback"}]': finished 2024-02-05T08:04:49.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:48 smithi131 ceph-mon[41654]: osdmap e654: 8 total, 8 up, 8 in 2024-02-05T08:04:49.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:49.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:49 smithi157 ceph-mon[33345]: osdmap e655: 8 total, 8 up, 8 in 2024-02-05T08:04:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130"}]: dispatch 2024-02-05T08:04:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130"}]: dispatch 2024-02-05T08:04:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:49 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:50.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:50.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:49 smithi131 ceph-mon[41654]: osdmap e655: 8 total, 8 up, 8 in 2024-02-05T08:04:50.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130"}]: dispatch 2024-02-05T08:04:50.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130"}]: dispatch 2024-02-05T08:04:50.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:49 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:50.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:04:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:49 smithi131 ceph-mon[29507]: osdmap e655: 8 total, 8 up, 8 in 2024-02-05T08:04:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130"}]: dispatch 2024-02-05T08:04:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130"}]: dispatch 2024-02-05T08:04:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:49 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:51.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:50 smithi157 ceph-mon[33345]: pgmap v1016: 268 pgs: 268 active+clean; 582 KiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T08:04:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:50 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:50 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130"}]': finished 2024-02-05T08:04:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:50 smithi157 ceph-mon[33345]: osdmap e656: 8 total, 8 up, 8 in 2024-02-05T08:04:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:50 smithi131 ceph-mon[41654]: pgmap v1016: 268 pgs: 268 active+clean; 582 KiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T08:04:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:50 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:50 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130"}]': finished 2024-02-05T08:04:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:50 smithi131 ceph-mon[41654]: osdmap e656: 8 total, 8 up, 8 in 2024-02-05T08:04:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:50 smithi131 ceph-mon[29507]: pgmap v1016: 268 pgs: 268 active+clean; 582 KiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T08:04:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:50 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:04:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:50 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-130"}]': finished 2024-02-05T08:04:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:50 smithi131 ceph-mon[29507]: osdmap e656: 8 total, 8 up, 8 in 2024-02-05T08:04:52.168 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:51 smithi131 ceph-mon[41654]: osdmap e657: 8 total, 8 up, 8 in 2024-02-05T08:04:52.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:51 smithi131 ceph-mon[29507]: osdmap e657: 8 total, 8 up, 8 in 2024-02-05T08:04:52.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:51 smithi157 ceph-mon[33345]: osdmap e657: 8 total, 8 up, 8 in 2024-02-05T08:04:53.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:52 smithi157 ceph-mon[33345]: pgmap v1019: 236 pgs: 236 active+clean; 582 KiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:53.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:52 smithi157 ceph-mon[33345]: osdmap e658: 8 total, 8 up, 8 in 2024-02-05T08:04:53.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:04:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:04:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:04:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:52 smithi131 ceph-mon[41654]: pgmap v1019: 236 pgs: 236 active+clean; 582 KiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:52 smithi131 ceph-mon[41654]: osdmap e658: 8 total, 8 up, 8 in 2024-02-05T08:04:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:52 smithi131 ceph-mon[29507]: pgmap v1019: 236 pgs: 236 active+clean; 582 KiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:04:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:52 smithi131 ceph-mon[29507]: osdmap e658: 8 total, 8 up, 8 in 2024-02-05T08:04:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:04:54.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:54.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:53 smithi157 ceph-mon[33345]: osdmap e659: 8 total, 8 up, 8 in 2024-02-05T08:04:54.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:54.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:54.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:54.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:53 smithi131 ceph-mon[41654]: osdmap e659: 8 total, 8 up, 8 in 2024-02-05T08:04:54.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:54.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:54.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:04:54.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:53 smithi131 ceph-mon[29507]: osdmap e659: 8 total, 8 up, 8 in 2024-02-05T08:04:54.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:54.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:04:55.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:54 smithi157 ceph-mon[33345]: pgmap v1022: 268 pgs: 268 active+clean; 582 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:04:55.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:55.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:54 smithi157 ceph-mon[33345]: osdmap e660: 8 total, 8 up, 8 in 2024-02-05T08:04:55.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:04:55.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:04:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:54 smithi131 ceph-mon[41654]: pgmap v1022: 268 pgs: 268 active+clean; 582 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:04:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:54 smithi131 ceph-mon[41654]: osdmap e660: 8 total, 8 up, 8 in 2024-02-05T08:04:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:04:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:04:55.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:54 smithi131 ceph-mon[29507]: pgmap v1022: 268 pgs: 268 active+clean; 582 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:04:55.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:04:55.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:54 smithi131 ceph-mon[29507]: osdmap e660: 8 total, 8 up, 8 in 2024-02-05T08:04:55.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:04:55.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:04:56.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]': finished 2024-02-05T08:04:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:55 smithi157 ceph-mon[33345]: osdmap e661: 8 total, 8 up, 8 in 2024-02-05T08:04:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-132", "mode": "writeback"}]: dispatch 2024-02-05T08:04:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-132", "mode": "writeback"}]: dispatch 2024-02-05T08:04:56.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]': finished 2024-02-05T08:04:56.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:55 smithi131 ceph-mon[41654]: osdmap e661: 8 total, 8 up, 8 in 2024-02-05T08:04:56.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-132", "mode": "writeback"}]: dispatch 2024-02-05T08:04:56.271 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-132", "mode": "writeback"}]: dispatch 2024-02-05T08:04:56.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]': finished 2024-02-05T08:04:56.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:55 smithi131 ceph-mon[29507]: osdmap e661: 8 total, 8 up, 8 in 2024-02-05T08:04:56.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-132", "mode": "writeback"}]: dispatch 2024-02-05T08:04:56.272 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-132", "mode": "writeback"}]: dispatch 2024-02-05T08:04:57.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:56 smithi157 ceph-mon[33345]: pgmap v1025: 268 pgs: 268 active+clean; 582 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:56 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-132", "mode": "writeback"}]': finished 2024-02-05T08:04:57.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:56 smithi157 ceph-mon[33345]: osdmap e662: 8 total, 8 up, 8 in 2024-02-05T08:04:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:56 smithi131 ceph-mon[29507]: pgmap v1025: 268 pgs: 268 active+clean; 582 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:56 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-132", "mode": "writeback"}]': finished 2024-02-05T08:04:57.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:56 smithi131 ceph-mon[29507]: osdmap e662: 8 total, 8 up, 8 in 2024-02-05T08:04:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:56 smithi131 ceph-mon[41654]: pgmap v1025: 268 pgs: 268 active+clean; 582 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:56 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:04:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-132", "mode": "writeback"}]': finished 2024-02-05T08:04:57.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:56 smithi131 ceph-mon[41654]: osdmap e662: 8 total, 8 up, 8 in 2024-02-05T08:04:58.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:57 smithi157 ceph-mon[33345]: osdmap e663: 8 total, 8 up, 8 in 2024-02-05T08:04:58.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:57 smithi131 ceph-mon[29507]: osdmap e663: 8 total, 8 up, 8 in 2024-02-05T08:04:58.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:57 smithi131 ceph-mon[41654]: osdmap e663: 8 total, 8 up, 8 in 2024-02-05T08:04:59.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:04:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:04:59.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:58 smithi157 ceph-mon[33345]: pgmap v1028: 268 pgs: 268 active+clean; 582 KiB data, 701 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:59.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:58 smithi157 ceph-mon[33345]: osdmap e664: 8 total, 8 up, 8 in 2024-02-05T08:04:59.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:59.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:59.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:58 smithi131 ceph-mon[29507]: pgmap v1028: 268 pgs: 268 active+clean; 582 KiB data, 701 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:59.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:58 smithi131 ceph-mon[29507]: osdmap e664: 8 total, 8 up, 8 in 2024-02-05T08:04:59.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:59.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:59.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:58 smithi131 ceph-mon[41654]: pgmap v1028: 268 pgs: 268 active+clean; 582 KiB data, 701 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:04:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:58 smithi131 ceph-mon[41654]: osdmap e664: 8 total, 8 up, 8 in 2024-02-05T08:04:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:04:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:00.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:05:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:59 smithi157 ceph-mon[33345]: osdmap e665: 8 total, 8 up, 8 in 2024-02-05T08:05:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:05:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:05:00.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:04:59 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:00.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:05:00.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:59 smithi131 ceph-mon[41654]: osdmap e665: 8 total, 8 up, 8 in 2024-02-05T08:05:00.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:05:00.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:05:00.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:04:59 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:00.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:05:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:59 smithi131 ceph-mon[29507]: osdmap e665: 8 total, 8 up, 8 in 2024-02-05T08:05:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:05:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:05:00.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:04:59 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:01.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:00 smithi157 ceph-mon[33345]: pgmap v1031: 268 pgs: 268 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T08:05:01.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]': finished 2024-02-05T08:05:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:00 smithi157 ceph-mon[33345]: osdmap e666: 8 total, 8 up, 8 in 2024-02-05T08:05:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:00 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:05:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:00 smithi131 ceph-mon[41654]: pgmap v1031: 268 pgs: 268 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T08:05:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]': finished 2024-02-05T08:05:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:00 smithi131 ceph-mon[41654]: osdmap e666: 8 total, 8 up, 8 in 2024-02-05T08:05:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:00 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:05:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:00 smithi131 ceph-mon[29507]: pgmap v1031: 268 pgs: 268 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T08:05:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-132"}]': finished 2024-02-05T08:05:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:00 smithi131 ceph-mon[29507]: osdmap e666: 8 total, 8 up, 8 in 2024-02-05T08:05:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:00 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:05:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:02.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:01 smithi157 ceph-mon[33345]: osdmap e667: 8 total, 8 up, 8 in 2024-02-05T08:05:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:02.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:01 smithi131 ceph-mon[41654]: osdmap e667: 8 total, 8 up, 8 in 2024-02-05T08:05:02.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:02.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:02.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:01 smithi131 ceph-mon[29507]: osdmap e667: 8 total, 8 up, 8 in 2024-02-05T08:05:02.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:02.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:05:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:05:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:05:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:02 smithi131 ceph-mon[41654]: pgmap v1034: 268 pgs: 268 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T08:05:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:05:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:02 smithi131 ceph-mon[41654]: osdmap e668: 8 total, 8 up, 8 in 2024-02-05T08:05:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:05:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:05:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:02 smithi131 ceph-mon[29507]: pgmap v1034: 268 pgs: 268 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T08:05:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:05:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:02 smithi131 ceph-mon[29507]: osdmap e668: 8 total, 8 up, 8 in 2024-02-05T08:05:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:05:03.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:05:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:02 smithi157 ceph-mon[33345]: pgmap v1034: 268 pgs: 268 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T08:05:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:05:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:02 smithi157 ceph-mon[33345]: osdmap e668: 8 total, 8 up, 8 in 2024-02-05T08:05:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:05:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132"}]: dispatch 2024-02-05T08:05:04.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:03 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:05:04.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132"}]': finished 2024-02-05T08:05:04.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:03 smithi131 ceph-mon[41654]: osdmap e669: 8 total, 8 up, 8 in 2024-02-05T08:05:04.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:03 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:05:04.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132"}]': finished 2024-02-05T08:05:04.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:03 smithi131 ceph-mon[29507]: osdmap e669: 8 total, 8 up, 8 in 2024-02-05T08:05:04.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:03 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:05:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-132"}]': finished 2024-02-05T08:05:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:03 smithi157 ceph-mon[33345]: osdmap e669: 8 total, 8 up, 8 in 2024-02-05T08:05:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:04 smithi131 ceph-mon[41654]: pgmap v1037: 268 pgs: 268 active+clean; 582 KiB data, 728 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T08:05:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:04 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:04 smithi131 ceph-mon[41654]: osdmap e670: 8 total, 8 up, 8 in 2024-02-05T08:05:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:04 smithi131 ceph-mon[29507]: pgmap v1037: 268 pgs: 268 active+clean; 582 KiB data, 728 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T08:05:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:04 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:05.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:04 smithi131 ceph-mon[29507]: osdmap e670: 8 total, 8 up, 8 in 2024-02-05T08:05:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:04 smithi157 ceph-mon[33345]: pgmap v1037: 268 pgs: 268 active+clean; 582 KiB data, 728 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T08:05:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:04 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:05.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:04 smithi157 ceph-mon[33345]: osdmap e670: 8 total, 8 up, 8 in 2024-02-05T08:05:06.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:05 smithi131 ceph-mon[41654]: osdmap e671: 8 total, 8 up, 8 in 2024-02-05T08:05:06.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:05 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:06.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:05 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:06.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:05 smithi131 ceph-mon[29507]: osdmap e671: 8 total, 8 up, 8 in 2024-02-05T08:05:06.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:05 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:06.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:05 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:06.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:05 smithi157 ceph-mon[33345]: osdmap e671: 8 total, 8 up, 8 in 2024-02-05T08:05:06.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:05 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:06.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:05 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:07.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[29507]: pgmap v1040: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 728 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:07.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:05:07.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[29507]: osdmap e672: 8 total, 8 up, 8 in 2024-02-05T08:05:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:05:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[29507]: osdmap e673: 8 total, 8 up, 8 in 2024-02-05T08:05:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-134"}]: dispatch 2024-02-05T08:05:07.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-134"}]: dispatch 2024-02-05T08:05:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[41654]: pgmap v1040: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 728 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:05:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[41654]: osdmap e672: 8 total, 8 up, 8 in 2024-02-05T08:05:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:07.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:07.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:05:07.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[41654]: osdmap e673: 8 total, 8 up, 8 in 2024-02-05T08:05:07.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-134"}]: dispatch 2024-02-05T08:05:07.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-134"}]: dispatch 2024-02-05T08:05:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:06 smithi157 ceph-mon[33345]: pgmap v1040: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 728 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:05:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:06 smithi157 ceph-mon[33345]: osdmap e672: 8 total, 8 up, 8 in 2024-02-05T08:05:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:05:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:06 smithi157 ceph-mon[33345]: osdmap e673: 8 total, 8 up, 8 in 2024-02-05T08:05:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-134"}]: dispatch 2024-02-05T08:05:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-134"}]: dispatch 2024-02-05T08:05:09.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:08 smithi131 ceph-mon[41654]: pgmap v1043: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 728 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:05:09.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-134"}]': finished 2024-02-05T08:05:09.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:08 smithi131 ceph-mon[41654]: osdmap e674: 8 total, 8 up, 8 in 2024-02-05T08:05:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-134", "mode": "writeback"}]: dispatch 2024-02-05T08:05:09.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-134", "mode": "writeback"}]: dispatch 2024-02-05T08:05:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:08 smithi131 ceph-mon[29507]: pgmap v1043: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 728 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:05:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-134"}]': finished 2024-02-05T08:05:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:08 smithi131 ceph-mon[29507]: osdmap e674: 8 total, 8 up, 8 in 2024-02-05T08:05:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-134", "mode": "writeback"}]: dispatch 2024-02-05T08:05:09.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-134", "mode": "writeback"}]: dispatch 2024-02-05T08:05:09.055 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:08 smithi157 ceph-mon[33345]: pgmap v1043: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 728 MiB used, 715 GiB / 715 GiB avail 2024-02-05T08:05:09.055 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-134"}]': finished 2024-02-05T08:05:09.055 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:08 smithi157 ceph-mon[33345]: osdmap e674: 8 total, 8 up, 8 in 2024-02-05T08:05:09.056 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-134", "mode": "writeback"}]: dispatch 2024-02-05T08:05:09.056 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-134", "mode": "writeback"}]: dispatch 2024-02-05T08:05:09.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:05:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:05:10.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:09 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:05:10.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-134", "mode": "writeback"}]': finished 2024-02-05T08:05:10.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:09 smithi131 ceph-mon[41654]: osdmap e675: 8 total, 8 up, 8 in 2024-02-05T08:05:10.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:10.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:09 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:09 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:05:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-134", "mode": "writeback"}]': finished 2024-02-05T08:05:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:09 smithi131 ceph-mon[29507]: osdmap e675: 8 total, 8 up, 8 in 2024-02-05T08:05:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:10.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:09 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:09 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:05:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-134", "mode": "writeback"}]': finished 2024-02-05T08:05:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:09 smithi157 ceph-mon[33345]: osdmap e675: 8 total, 8 up, 8 in 2024-02-05T08:05:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:10.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:10.222 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:09 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:11.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:10 smithi131 ceph-mon[41654]: pgmap v1046: 268 pgs: 268 active+clean; 582 KiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:11.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:05:11.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:10 smithi131 ceph-mon[41654]: osdmap e676: 8 total, 8 up, 8 in 2024-02-05T08:05:11.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:10 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134"}]: dispatch 2024-02-05T08:05:11.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134"}]: dispatch 2024-02-05T08:05:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:10 smithi131 ceph-mon[29507]: pgmap v1046: 268 pgs: 268 active+clean; 582 KiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:05:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:10 smithi131 ceph-mon[29507]: osdmap e676: 8 total, 8 up, 8 in 2024-02-05T08:05:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:10 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134"}]: dispatch 2024-02-05T08:05:11.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134"}]: dispatch 2024-02-05T08:05:11.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:10 smithi157 ceph-mon[33345]: pgmap v1046: 268 pgs: 268 active+clean; 582 KiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:11.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:05:11.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:10 smithi157 ceph-mon[33345]: osdmap e676: 8 total, 8 up, 8 in 2024-02-05T08:05:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:10 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134"}]: dispatch 2024-02-05T08:05:11.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134"}]: dispatch 2024-02-05T08:05:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:11 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:05:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:11 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134"}]': finished 2024-02-05T08:05:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:11 smithi157 ceph-mon[33345]: osdmap e677: 8 total, 8 up, 8 in 2024-02-05T08:05:12.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:11 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:12.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:11 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:05:12.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:11 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134"}]': finished 2024-02-05T08:05:12.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:11 smithi131 ceph-mon[41654]: osdmap e677: 8 total, 8 up, 8 in 2024-02-05T08:05:12.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:11 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:12.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:11 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:05:12.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:11 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-134"}]': finished 2024-02-05T08:05:12.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:11 smithi131 ceph-mon[29507]: osdmap e677: 8 total, 8 up, 8 in 2024-02-05T08:05:12.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:11 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:13.017 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:05:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:05:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:05:13.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:12 smithi131 ceph-mon[41654]: pgmap v1049: 268 pgs: 268 active+clean; 582 KiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:13.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:12 smithi131 ceph-mon[41654]: osdmap e678: 8 total, 8 up, 8 in 2024-02-05T08:05:13.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:12 smithi131 ceph-mon[29507]: pgmap v1049: 268 pgs: 268 active+clean; 582 KiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:13.018 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:12 smithi131 ceph-mon[29507]: osdmap e678: 8 total, 8 up, 8 in 2024-02-05T08:05:13.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:12 smithi157 ceph-mon[33345]: pgmap v1049: 268 pgs: 268 active+clean; 582 KiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:13.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:12 smithi157 ceph-mon[33345]: osdmap e678: 8 total, 8 up, 8 in 2024-02-05T08:05:14.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:13 smithi157 ceph-mon[33345]: osdmap e679: 8 total, 8 up, 8 in 2024-02-05T08:05:14.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:13 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:14.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:13 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:14.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:13 smithi131 ceph-mon[41654]: osdmap e679: 8 total, 8 up, 8 in 2024-02-05T08:05:14.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:13 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:14.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:13 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:14.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:13 smithi131 ceph-mon[29507]: osdmap e679: 8 total, 8 up, 8 in 2024-02-05T08:05:14.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:13 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:14.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:13 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:15.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:14 smithi157 ceph-mon[33345]: pgmap v1052: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:05:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:05:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:14 smithi157 ceph-mon[33345]: osdmap e680: 8 total, 8 up, 8 in 2024-02-05T08:05:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:15.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:15.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:14 smithi131 ceph-mon[41654]: pgmap v1052: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:05:15.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:05:15.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:14 smithi131 ceph-mon[41654]: osdmap e680: 8 total, 8 up, 8 in 2024-02-05T08:05:15.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:15.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:14 smithi131 ceph-mon[29507]: pgmap v1052: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:05:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:05:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:14 smithi131 ceph-mon[29507]: osdmap e680: 8 total, 8 up, 8 in 2024-02-05T08:05:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:16.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:05:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:15 smithi157 ceph-mon[33345]: osdmap e681: 8 total, 8 up, 8 in 2024-02-05T08:05:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-136"}]: dispatch 2024-02-05T08:05:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:15 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-136"}]: dispatch 2024-02-05T08:05:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:15 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:05:16.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:16.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:05:16.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:15 smithi131 ceph-mon[41654]: osdmap e681: 8 total, 8 up, 8 in 2024-02-05T08:05:16.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-136"}]: dispatch 2024-02-05T08:05:16.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:15 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-136"}]: dispatch 2024-02-05T08:05:16.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:15 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:05:16.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:16.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:05:16.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:15 smithi131 ceph-mon[29507]: osdmap e681: 8 total, 8 up, 8 in 2024-02-05T08:05:16.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-136"}]: dispatch 2024-02-05T08:05:16.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:15 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-136"}]: dispatch 2024-02-05T08:05:16.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:15 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:05:16.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:17.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:16 smithi157 ceph-mon[33345]: pgmap v1055: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:05:17.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-136"}]': finished 2024-02-05T08:05:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:16 smithi157 ceph-mon[33345]: osdmap e682: 8 total, 8 up, 8 in 2024-02-05T08:05:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:16 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-136", "mode": "writeback"}]: dispatch 2024-02-05T08:05:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-136", "mode": "writeback"}]: dispatch 2024-02-05T08:05:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:16 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:05:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:16 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-136", "mode": "writeback"}]': finished 2024-02-05T08:05:17.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:16 smithi157 ceph-mon[33345]: osdmap e683: 8 total, 8 up, 8 in 2024-02-05T08:05:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:16 smithi131 ceph-mon[29507]: pgmap v1055: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:05:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-136"}]': finished 2024-02-05T08:05:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:16 smithi131 ceph-mon[29507]: osdmap e682: 8 total, 8 up, 8 in 2024-02-05T08:05:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:16 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-136", "mode": "writeback"}]: dispatch 2024-02-05T08:05:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-136", "mode": "writeback"}]: dispatch 2024-02-05T08:05:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:16 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:05:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:16 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-136", "mode": "writeback"}]': finished 2024-02-05T08:05:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:16 smithi131 ceph-mon[29507]: osdmap e683: 8 total, 8 up, 8 in 2024-02-05T08:05:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:16 smithi131 ceph-mon[41654]: pgmap v1055: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:05:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-136"}]': finished 2024-02-05T08:05:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:16 smithi131 ceph-mon[41654]: osdmap e682: 8 total, 8 up, 8 in 2024-02-05T08:05:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:16 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-136", "mode": "writeback"}]: dispatch 2024-02-05T08:05:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-136", "mode": "writeback"}]: dispatch 2024-02-05T08:05:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:16 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:05:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:16 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-136", "mode": "writeback"}]': finished 2024-02-05T08:05:17.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:16 smithi131 ceph-mon[41654]: osdmap e683: 8 total, 8 up, 8 in 2024-02-05T08:05:17.754 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ]c 2024-02-05T08:05:17.754 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.754 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.754 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.754 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.754 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.754 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.754 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.754 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.754 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.755 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.755 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.755 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.755 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.755 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.755 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.755 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.755 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.755 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.755 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.755 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.755 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.755 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.755 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.755 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.755 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.756 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.757 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.758 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.759 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.760 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.760 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.760 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.760 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.760 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.760 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.760 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.760 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.760 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:17.760 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:05:18.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:18.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:17 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:18.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:18.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:17 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:18.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:18.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:17 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:19.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:05:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:05:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:18 smithi157 ceph-mon[33345]: pgmap v1058: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 751 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:05:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:05:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:18 smithi157 ceph-mon[33345]: osdmap e684: 8 total, 8 up, 8 in 2024-02-05T08:05:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136"}]: dispatch 2024-02-05T08:05:19.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:18 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136"}]: dispatch 2024-02-05T08:05:19.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:18 smithi131 ceph-mon[41654]: pgmap v1058: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 751 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:05:19.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:05:19.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:18 smithi131 ceph-mon[41654]: osdmap e684: 8 total, 8 up, 8 in 2024-02-05T08:05:19.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136"}]: dispatch 2024-02-05T08:05:19.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:18 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136"}]: dispatch 2024-02-05T08:05:19.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:18 smithi131 ceph-mon[29507]: pgmap v1058: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 751 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:05:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:05:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:18 smithi131 ceph-mon[29507]: osdmap e684: 8 total, 8 up, 8 in 2024-02-05T08:05:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136"}]: dispatch 2024-02-05T08:05:19.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:18 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136"}]: dispatch 2024-02-05T08:05:20.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:19 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:05:20.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:19 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136"}]': finished 2024-02-05T08:05:20.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:19 smithi157 ceph-mon[33345]: osdmap e685: 8 total, 8 up, 8 in 2024-02-05T08:05:20.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:19 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:20.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:19 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:05:20.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:19 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136"}]': finished 2024-02-05T08:05:20.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:19 smithi131 ceph-mon[29507]: osdmap e685: 8 total, 8 up, 8 in 2024-02-05T08:05:20.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:19 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:20.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:19 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:05:20.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:19 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-136"}]': finished 2024-02-05T08:05:20.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:19 smithi131 ceph-mon[41654]: osdmap e685: 8 total, 8 up, 8 in 2024-02-05T08:05:20.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:19 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:21.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:20 smithi157 ceph-mon[33345]: pgmap v1061: 268 pgs: 268 active+clean; 4.4 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-02-05T08:05:21.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:20 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:21.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:20 smithi157 ceph-mon[33345]: osdmap e686: 8 total, 8 up, 8 in 2024-02-05T08:05:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:20 smithi131 ceph-mon[29507]: pgmap v1061: 268 pgs: 268 active+clean; 4.4 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-02-05T08:05:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:20 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:20 smithi131 ceph-mon[29507]: osdmap e686: 8 total, 8 up, 8 in 2024-02-05T08:05:21.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:20 smithi131 ceph-mon[41654]: pgmap v1061: 268 pgs: 268 active+clean; 4.4 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-02-05T08:05:21.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:20 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:21.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:20 smithi131 ceph-mon[41654]: osdmap e686: 8 total, 8 up, 8 in 2024-02-05T08:05:22.168 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:21 smithi131 ceph-mon[41654]: osdmap e687: 8 total, 8 up, 8 in 2024-02-05T08:05:22.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:22.169 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:22.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:21 smithi131 ceph-mon[29507]: osdmap e687: 8 total, 8 up, 8 in 2024-02-05T08:05:22.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:22.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:22.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:21 smithi157 ceph-mon[33345]: osdmap e687: 8 total, 8 up, 8 in 2024-02-05T08:05:22.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:22.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:23.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:22 smithi157 ceph-mon[33345]: pgmap v1064: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-05T08:05:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:05:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:22 smithi157 ceph-mon[33345]: osdmap e688: 8 total, 8 up, 8 in 2024-02-05T08:05:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:22 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:05:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:22 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:05:23.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:22 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:05:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:05:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:05:23.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-mon[29507]: pgmap v1064: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-05T08:05:23.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:05:23.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-mon[29507]: osdmap e688: 8 total, 8 up, 8 in 2024-02-05T08:05:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:05:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:05:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:05:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-mon[41654]: pgmap v1064: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-05T08:05:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:05:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-mon[41654]: osdmap e688: 8 total, 8 up, 8 in 2024-02-05T08:05:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:05:23.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:05:23.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:22 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:05:24.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:05:24.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:23 smithi157 ceph-mon[33345]: osdmap e689: 8 total, 8 up, 8 in 2024-02-05T08:05:24.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:23 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:05:24.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:05:24.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:23 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:05:24.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:23 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:05:24.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:23 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:05:24.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:23 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_count","val": "2"}]': finished 2024-02-05T08:05:24.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:05:24.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:23 smithi131 ceph-mon[41654]: osdmap e689: 8 total, 8 up, 8 in 2024-02-05T08:05:24.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:23 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:05:24.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:05:24.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:23 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:05:24.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:23 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:05:24.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:23 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:05:24.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:23 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_count","val": "2"}]': finished 2024-02-05T08:05:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:05:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:23 smithi131 ceph-mon[29507]: osdmap e689: 8 total, 8 up, 8 in 2024-02-05T08:05:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:23 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:05:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:05:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:23 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:05:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:23 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:05:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:23 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:05:24.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:23 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_count","val": "2"}]': finished 2024-02-05T08:05:25.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:24 smithi157 ceph-mon[33345]: pgmap v1067: 268 pgs: 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:25.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:24 smithi157 ceph-mon[33345]: osdmap e690: 8 total, 8 up, 8 in 2024-02-05T08:05:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:24 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:05:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:05:25.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:05:25.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:24 smithi131 ceph-mon[29507]: pgmap v1067: 268 pgs: 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:25.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:24 smithi131 ceph-mon[29507]: osdmap e690: 8 total, 8 up, 8 in 2024-02-05T08:05:25.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:05:25.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:05:25.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:05:25.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:24 smithi131 ceph-mon[41654]: pgmap v1067: 268 pgs: 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:25.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:24 smithi131 ceph-mon[41654]: osdmap e690: 8 total, 8 up, 8 in 2024-02-05T08:05:25.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:24 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:05:25.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:05:25.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:05:26.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:25 smithi157 ceph-mon[33345]: osdmap e691: 8 total, 8 up, 8 in 2024-02-05T08:05:26.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T08:05:26.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T08:05:26.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:25 smithi131 ceph-mon[41654]: osdmap e691: 8 total, 8 up, 8 in 2024-02-05T08:05:26.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T08:05:26.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T08:05:26.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:25 smithi131 ceph-mon[29507]: osdmap e691: 8 total, 8 up, 8 in 2024-02-05T08:05:26.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T08:05:26.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T08:05:27.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:26 smithi157 ceph-mon[33345]: pgmap v1070: 268 pgs: 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:27.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T08:05:27.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:26 smithi157 ceph-mon[33345]: osdmap e692: 8 total, 8 up, 8 in 2024-02-05T08:05:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:26 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:26 smithi131 ceph-mon[41654]: pgmap v1070: 268 pgs: 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T08:05:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:26 smithi131 ceph-mon[41654]: osdmap e692: 8 total, 8 up, 8 in 2024-02-05T08:05:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:26 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:27.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:26 smithi131 ceph-mon[29507]: pgmap v1070: 268 pgs: 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T08:05:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:26 smithi131 ceph-mon[29507]: osdmap e692: 8 total, 8 up, 8 in 2024-02-05T08:05:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:26 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:28.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:27 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:28.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:28.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:27 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138"}]: dispatch 2024-02-05T08:05:28.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138"}]: dispatch 2024-02-05T08:05:28.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:27 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:28.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:28.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:27 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138"}]: dispatch 2024-02-05T08:05:28.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138"}]: dispatch 2024-02-05T08:05:28.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:27 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:28.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:28.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:27 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138"}]: dispatch 2024-02-05T08:05:28.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138"}]: dispatch 2024-02-05T08:05:29.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:28 smithi157 ceph-mon[33345]: pgmap v1072: 268 pgs: 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:05:29.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138"}]': finished 2024-02-05T08:05:29.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:28 smithi157 ceph-mon[33345]: osdmap e693: 8 total, 8 up, 8 in 2024-02-05T08:05:29.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:05:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:05:29.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:28 smithi131 ceph-mon[41654]: pgmap v1072: 268 pgs: 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:05:29.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138"}]': finished 2024-02-05T08:05:29.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:28 smithi131 ceph-mon[41654]: osdmap e693: 8 total, 8 up, 8 in 2024-02-05T08:05:29.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:28 smithi131 ceph-mon[29507]: pgmap v1072: 268 pgs: 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:05:29.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-138"}]': finished 2024-02-05T08:05:29.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:28 smithi131 ceph-mon[29507]: osdmap e693: 8 total, 8 up, 8 in 2024-02-05T08:05:30.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:29 smithi157 ceph-mon[33345]: osdmap e694: 8 total, 8 up, 8 in 2024-02-05T08:05:30.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:29 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:30.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:29 smithi131 ceph-mon[41654]: osdmap e694: 8 total, 8 up, 8 in 2024-02-05T08:05:30.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:29 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:30.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:29 smithi131 ceph-mon[29507]: osdmap e694: 8 total, 8 up, 8 in 2024-02-05T08:05:30.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:29 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:30 smithi157 ceph-mon[33345]: pgmap v1075: 236 pgs: 236 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:05:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:30 smithi157 ceph-mon[33345]: osdmap e695: 8 total, 8 up, 8 in 2024-02-05T08:05:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:30 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:30 smithi131 ceph-mon[41654]: pgmap v1075: 236 pgs: 236 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:05:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:30 smithi131 ceph-mon[41654]: osdmap e695: 8 total, 8 up, 8 in 2024-02-05T08:05:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:30 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:30 smithi131 ceph-mon[29507]: pgmap v1075: 236 pgs: 236 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:05:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:30 smithi131 ceph-mon[29507]: osdmap e695: 8 total, 8 up, 8 in 2024-02-05T08:05:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:30 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:32.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:31 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:05:32.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:31 smithi157 ceph-mon[33345]: osdmap e696: 8 total, 8 up, 8 in 2024-02-05T08:05:32.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:31 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:32.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:31 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:32.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:31 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:05:32.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:31 smithi131 ceph-mon[41654]: osdmap e696: 8 total, 8 up, 8 in 2024-02-05T08:05:32.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:31 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:32.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:31 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:32.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:31 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:05:32.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:31 smithi131 ceph-mon[29507]: osdmap e696: 8 total, 8 up, 8 in 2024-02-05T08:05:32.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:31 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:32.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:31 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:33.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:32 smithi157 ceph-mon[33345]: pgmap v1078: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:05:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:32 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T08:05:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:32 smithi157 ceph-mon[33345]: osdmap e697: 8 total, 8 up, 8 in 2024-02-05T08:05:33.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T08:05:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:05:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:05:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:05:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:32 smithi131 ceph-mon[41654]: pgmap v1078: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:05:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:32 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T08:05:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:32 smithi131 ceph-mon[41654]: osdmap e697: 8 total, 8 up, 8 in 2024-02-05T08:05:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T08:05:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:32 smithi131 ceph-mon[29507]: pgmap v1078: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:05:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:32 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T08:05:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:32 smithi131 ceph-mon[29507]: osdmap e697: 8 total, 8 up, 8 in 2024-02-05T08:05:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T08:05:34.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_count","val": "3"}]': finished 2024-02-05T08:05:34.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:33 smithi157 ceph-mon[33345]: osdmap e698: 8 total, 8 up, 8 in 2024-02-05T08:05:34.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:33 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T08:05:34.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T08:05:34.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_count","val": "3"}]': finished 2024-02-05T08:05:34.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:33 smithi131 ceph-mon[41654]: osdmap e698: 8 total, 8 up, 8 in 2024-02-05T08:05:34.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:33 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T08:05:34.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T08:05:34.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_count","val": "3"}]': finished 2024-02-05T08:05:34.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:33 smithi131 ceph-mon[29507]: osdmap e698: 8 total, 8 up, 8 in 2024-02-05T08:05:34.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:33 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T08:05:34.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T08:05:35.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:34 smithi157 ceph-mon[33345]: pgmap v1081: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_period","val": "3"}]': finished 2024-02-05T08:05:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:34 smithi157 ceph-mon[33345]: osdmap e699: 8 total, 8 up, 8 in 2024-02-05T08:05:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:05:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:05:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:34 smithi131 ceph-mon[41654]: pgmap v1081: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_period","val": "3"}]': finished 2024-02-05T08:05:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:34 smithi131 ceph-mon[41654]: osdmap e699: 8 total, 8 up, 8 in 2024-02-05T08:05:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:05:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:05:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:34 smithi131 ceph-mon[29507]: pgmap v1081: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_period","val": "3"}]': finished 2024-02-05T08:05:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:34 smithi131 ceph-mon[29507]: osdmap e699: 8 total, 8 up, 8 in 2024-02-05T08:05:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:05:35.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:05:36.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T08:05:36.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:35 smithi157 ceph-mon[33345]: osdmap e700: 8 total, 8 up, 8 in 2024-02-05T08:05:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T08:05:36.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T08:05:36.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T08:05:36.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:35 smithi131 ceph-mon[41654]: osdmap e700: 8 total, 8 up, 8 in 2024-02-05T08:05:36.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T08:05:36.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T08:05:36.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T08:05:36.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:35 smithi131 ceph-mon[29507]: osdmap e700: 8 total, 8 up, 8 in 2024-02-05T08:05:36.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T08:05:36.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T08:05:37.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:36 smithi157 ceph-mon[33345]: pgmap v1084: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:36 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-05T08:05:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:36 smithi157 ceph-mon[33345]: osdmap e701: 8 total, 8 up, 8 in 2024-02-05T08:05:37.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:36 smithi131 ceph-mon[41654]: pgmap v1084: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:37.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:36 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-05T08:05:37.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:36 smithi131 ceph-mon[41654]: osdmap e701: 8 total, 8 up, 8 in 2024-02-05T08:05:37.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:36 smithi131 ceph-mon[29507]: pgmap v1084: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:37.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:36 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-05T08:05:37.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:36 smithi131 ceph-mon[29507]: osdmap e701: 8 total, 8 up, 8 in 2024-02-05T08:05:39.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:38 smithi157 ceph-mon[33345]: pgmap v1086: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:05:39.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:05:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:05:39.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:38 smithi131 ceph-mon[41654]: pgmap v1086: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:05:39.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:38 smithi131 ceph-mon[29507]: pgmap v1086: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:05:40.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:39 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:40.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:39 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:40.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:39 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:40 smithi157 ceph-mon[33345]: pgmap v1087: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:05:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:40 smithi131 ceph-mon[41654]: pgmap v1087: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:05:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:40 smithi131 ceph-mon[29507]: pgmap v1087: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:05:43.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:42 smithi157 ceph-mon[33345]: pgmap v1088: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 667 B/s rd, 3.1 KiB/s wr, 1 op/s 2024-02-05T08:05:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:42 smithi131 ceph-mon[41654]: pgmap v1088: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 667 B/s rd, 3.1 KiB/s wr, 1 op/s 2024-02-05T08:05:43.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:42 smithi131 ceph-mon[29507]: pgmap v1088: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 667 B/s rd, 3.1 KiB/s wr, 1 op/s 2024-02-05T08:05:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:05:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:05:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:05:45.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:44 smithi157 ceph-mon[33345]: pgmap v1089: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.6 KiB/s wr, 2 op/s 2024-02-05T08:05:45.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:44 smithi131 ceph-mon[41654]: pgmap v1089: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.6 KiB/s wr, 2 op/s 2024-02-05T08:05:45.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:44 smithi131 ceph-mon[29507]: pgmap v1089: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.6 KiB/s wr, 2 op/s 2024-02-05T08:05:46.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:45 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:45 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:46.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:45 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:47.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:46 smithi157 ceph-mon[33345]: pgmap v1090: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-02-05T08:05:47.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:46 smithi131 ceph-mon[41654]: pgmap v1090: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-02-05T08:05:47.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:46 smithi131 ceph-mon[29507]: pgmap v1090: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-02-05T08:05:49.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:05:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:05:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:48 smithi157 ceph-mon[33345]: pgmap v1091: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 878 B/s rd, 5.7 KiB/s wr, 1 op/s 2024-02-05T08:05:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140"}]: dispatch 2024-02-05T08:05:49.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140"}]: dispatch 2024-02-05T08:05:49.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:48 smithi131 ceph-mon[41654]: pgmap v1091: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 878 B/s rd, 5.7 KiB/s wr, 1 op/s 2024-02-05T08:05:49.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:49.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:49.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140"}]: dispatch 2024-02-05T08:05:49.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140"}]: dispatch 2024-02-05T08:05:49.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:48 smithi131 ceph-mon[29507]: pgmap v1091: 268 pgs: 268 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 878 B/s rd, 5.7 KiB/s wr, 1 op/s 2024-02-05T08:05:49.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:49.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:05:49.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140"}]: dispatch 2024-02-05T08:05:49.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140"}]: dispatch 2024-02-05T08:05:50.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140"}]': finished 2024-02-05T08:05:50.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:49 smithi131 ceph-mon[41654]: osdmap e702: 8 total, 8 up, 8 in 2024-02-05T08:05:50.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:49 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140"}]': finished 2024-02-05T08:05:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:49 smithi131 ceph-mon[29507]: osdmap e702: 8 total, 8 up, 8 in 2024-02-05T08:05:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:49 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:50.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-140"}]': finished 2024-02-05T08:05:50.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:49 smithi157 ceph-mon[33345]: osdmap e702: 8 total, 8 up, 8 in 2024-02-05T08:05:50.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:49 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:50 smithi131 ceph-mon[41654]: pgmap v1093: 268 pgs: 268 active+clean; 4.5 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.4 KiB/s wr, 2 op/s 2024-02-05T08:05:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:50 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:50 smithi131 ceph-mon[41654]: osdmap e703: 8 total, 8 up, 8 in 2024-02-05T08:05:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:50 smithi131 ceph-mon[29507]: pgmap v1093: 268 pgs: 268 active+clean; 4.5 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.4 KiB/s wr, 2 op/s 2024-02-05T08:05:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:50 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:50 smithi131 ceph-mon[29507]: osdmap e703: 8 total, 8 up, 8 in 2024-02-05T08:05:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:50 smithi157 ceph-mon[33345]: pgmap v1093: 268 pgs: 268 active+clean; 4.5 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.4 KiB/s wr, 2 op/s 2024-02-05T08:05:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:50 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:50 smithi157 ceph-mon[33345]: osdmap e703: 8 total, 8 up, 8 in 2024-02-05T08:05:52.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:51 smithi131 ceph-mon[41654]: osdmap e704: 8 total, 8 up, 8 in 2024-02-05T08:05:52.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:52.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:52.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:51 smithi131 ceph-mon[29507]: osdmap e704: 8 total, 8 up, 8 in 2024-02-05T08:05:52.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:52.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:52.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:51 smithi157 ceph-mon[33345]: osdmap e704: 8 total, 8 up, 8 in 2024-02-05T08:05:52.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:52.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:05:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:05:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:05:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:05:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:53 smithi131 ceph-mon[41654]: pgmap v1096: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:05:53.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:53 smithi131 ceph-mon[41654]: osdmap e705: 8 total, 8 up, 8 in 2024-02-05T08:05:53.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:53.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:53 smithi131 ceph-mon[29507]: pgmap v1096: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:05:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:53 smithi131 ceph-mon[29507]: osdmap e705: 8 total, 8 up, 8 in 2024-02-05T08:05:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:53 smithi157 ceph-mon[33345]: pgmap v1096: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:05:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:53 smithi157 ceph-mon[33345]: osdmap e705: 8 total, 8 up, 8 in 2024-02-05T08:05:53.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:53.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:05:54.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:05:54.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:54 smithi131 ceph-mon[41654]: osdmap e706: 8 total, 8 up, 8 in 2024-02-05T08:05:54.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:05:54.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:05:54.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:05:54.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:54 smithi131 ceph-mon[29507]: osdmap e706: 8 total, 8 up, 8 in 2024-02-05T08:05:54.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:05:54.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:05:54.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:05:54.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:54 smithi157 ceph-mon[33345]: osdmap e706: 8 total, 8 up, 8 in 2024-02-05T08:05:54.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:05:54.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:05:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:55 smithi157 ceph-mon[33345]: pgmap v1099: 268 pgs: 268 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:05:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-142"}]': finished 2024-02-05T08:05:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:55 smithi157 ceph-mon[33345]: osdmap e707: 8 total, 8 up, 8 in 2024-02-05T08:05:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-142", "mode": "writeback"}]: dispatch 2024-02-05T08:05:55.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-142", "mode": "writeback"}]: dispatch 2024-02-05T08:05:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:55 smithi131 ceph-mon[41654]: pgmap v1099: 268 pgs: 268 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:05:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-142"}]': finished 2024-02-05T08:05:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:55 smithi131 ceph-mon[41654]: osdmap e707: 8 total, 8 up, 8 in 2024-02-05T08:05:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-142", "mode": "writeback"}]: dispatch 2024-02-05T08:05:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-142", "mode": "writeback"}]: dispatch 2024-02-05T08:05:55.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:55 smithi131 ceph-mon[29507]: pgmap v1099: 268 pgs: 268 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:05:55.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-142"}]': finished 2024-02-05T08:05:55.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:55 smithi131 ceph-mon[29507]: osdmap e707: 8 total, 8 up, 8 in 2024-02-05T08:05:55.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-142", "mode": "writeback"}]: dispatch 2024-02-05T08:05:55.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-142", "mode": "writeback"}]: dispatch 2024-02-05T08:05:56.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:56 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:05:56.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-142", "mode": "writeback"}]': finished 2024-02-05T08:05:56.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:56 smithi157 ceph-mon[33345]: osdmap e708: 8 total, 8 up, 8 in 2024-02-05T08:05:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:05:56.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:56 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:05:56.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:56 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:05:56.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-142", "mode": "writeback"}]': finished 2024-02-05T08:05:56.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:56 smithi131 ceph-mon[41654]: osdmap e708: 8 total, 8 up, 8 in 2024-02-05T08:05:56.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:05:56.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:56 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:05:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:56 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:05:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-142", "mode": "writeback"}]': finished 2024-02-05T08:05:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:56 smithi131 ceph-mon[29507]: osdmap e708: 8 total, 8 up, 8 in 2024-02-05T08:05:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:05:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:56 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:05:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:57 smithi157 ceph-mon[33345]: pgmap v1102: 268 pgs: 268 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T08:05:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:57 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_count","val": "2"}]': finished 2024-02-05T08:05:57.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:57 smithi157 ceph-mon[33345]: osdmap e709: 8 total, 8 up, 8 in 2024-02-05T08:05:57.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:57 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:05:57.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:57 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:05:57.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:57 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:57 smithi131 ceph-mon[41654]: pgmap v1102: 268 pgs: 268 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T08:05:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:57 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_count","val": "2"}]': finished 2024-02-05T08:05:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:57 smithi131 ceph-mon[41654]: osdmap e709: 8 total, 8 up, 8 in 2024-02-05T08:05:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:57 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:05:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:57 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:05:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:57 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:57.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:57 smithi131 ceph-mon[29507]: pgmap v1102: 268 pgs: 268 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T08:05:57.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:57 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_count","val": "2"}]': finished 2024-02-05T08:05:57.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:57 smithi131 ceph-mon[29507]: osdmap e709: 8 total, 8 up, 8 in 2024-02-05T08:05:57.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:57 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:05:57.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:57 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:05:57.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:57 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:05:58.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:05:58.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:58 smithi157 ceph-mon[33345]: osdmap e710: 8 total, 8 up, 8 in 2024-02-05T08:05:58.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:05:58.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:05:58.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:58 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:05:58.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T08:05:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:05:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:58 smithi131 ceph-mon[41654]: osdmap e710: 8 total, 8 up, 8 in 2024-02-05T08:05:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:05:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:05:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:58 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:05:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T08:05:58.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:05:58.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:58 smithi131 ceph-mon[29507]: osdmap e710: 8 total, 8 up, 8 in 2024-02-05T08:05:58.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:05:58.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:05:58.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:58 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:05:58.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T08:05:59.372 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:05:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:05:59.372 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:59 smithi157 ceph-mon[33345]: pgmap v1105: 268 pgs: 268 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:05:59.372 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:59 smithi157 ceph-mon[33345]: osdmap e711: 8 total, 8 up, 8 in 2024-02-05T08:05:59.372 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T08:05:59.372 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T08:05:59.372 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T08:05:59.372 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:59 smithi157 ceph-mon[33345]: osdmap e712: 8 total, 8 up, 8 in 2024-02-05T08:05:59.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T08:05:59.373 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:05:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T08:05:59.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:59 smithi131 ceph-mon[41654]: pgmap v1105: 268 pgs: 268 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:05:59.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:59 smithi131 ceph-mon[41654]: osdmap e711: 8 total, 8 up, 8 in 2024-02-05T08:05:59.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T08:05:59.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T08:05:59.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T08:05:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:59 smithi131 ceph-mon[41654]: osdmap e712: 8 total, 8 up, 8 in 2024-02-05T08:05:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T08:05:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:05:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T08:05:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:59 smithi131 ceph-mon[29507]: pgmap v1105: 268 pgs: 268 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:05:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:59 smithi131 ceph-mon[29507]: osdmap e711: 8 total, 8 up, 8 in 2024-02-05T08:05:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T08:05:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T08:05:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T08:05:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:59 smithi131 ceph-mon[29507]: osdmap e712: 8 total, 8 up, 8 in 2024-02-05T08:05:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T08:05:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:05:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T08:06:00.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:00 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:00.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:00.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-05T08:06:00.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:00 smithi157 ceph-mon[33345]: osdmap e713: 8 total, 8 up, 8 in 2024-02-05T08:06:00.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T08:06:00.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T08:06:00.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:00 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:00.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:00.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-05T08:06:00.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:00 smithi131 ceph-mon[41654]: osdmap e713: 8 total, 8 up, 8 in 2024-02-05T08:06:00.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T08:06:00.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T08:06:00.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:00 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:00.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:00.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-05T08:06:00.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:00 smithi131 ceph-mon[29507]: osdmap e713: 8 total, 8 up, 8 in 2024-02-05T08:06:00.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T08:06:00.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T08:06:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:01 smithi157 ceph-mon[33345]: pgmap v1108: 268 pgs: 268 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-05T08:06:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:01 smithi157 ceph-mon[33345]: osdmap e714: 8 total, 8 up, 8 in 2024-02-05T08:06:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:01 smithi131 ceph-mon[41654]: pgmap v1108: 268 pgs: 268 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-05T08:06:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:01 smithi131 ceph-mon[41654]: osdmap e714: 8 total, 8 up, 8 in 2024-02-05T08:06:01.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:01 smithi131 ceph-mon[29507]: pgmap v1108: 268 pgs: 268 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:01.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-05T08:06:01.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:01 smithi131 ceph-mon[29507]: osdmap e714: 8 total, 8 up, 8 in 2024-02-05T08:06:02.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:02 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:02.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:02 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:02.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:02 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:02.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:02 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:02.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:02 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:02.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:02 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:06:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:06:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:06:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:03 smithi131 ceph-mon[41654]: pgmap v1111: 268 pgs: 268 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:06:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:03 smithi131 ceph-mon[41654]: osdmap e715: 8 total, 8 up, 8 in 2024-02-05T08:06:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:03 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:06:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:03 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:06:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:03 smithi131 ceph-mon[29507]: pgmap v1111: 268 pgs: 268 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:06:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:03 smithi131 ceph-mon[29507]: osdmap e715: 8 total, 8 up, 8 in 2024-02-05T08:06:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:03 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:06:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:03 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:06:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:03 smithi157 ceph-mon[33345]: pgmap v1111: 268 pgs: 268 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:06:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:03 smithi157 ceph-mon[33345]: osdmap e715: 8 total, 8 up, 8 in 2024-02-05T08:06:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:03 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:06:03.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:03 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:06:04.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142"}]': finished 2024-02-05T08:06:04.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:04 smithi157 ceph-mon[33345]: osdmap e716: 8 total, 8 up, 8 in 2024-02-05T08:06:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:04 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:06:04.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:04 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:06:04.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142"}]': finished 2024-02-05T08:06:04.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:04 smithi131 ceph-mon[41654]: osdmap e716: 8 total, 8 up, 8 in 2024-02-05T08:06:04.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:04.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:04.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:04 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:06:04.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:04 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:06:04.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142"}]': finished 2024-02-05T08:06:04.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:04 smithi131 ceph-mon[29507]: osdmap e716: 8 total, 8 up, 8 in 2024-02-05T08:06:04.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:04.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:04.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:04 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:06:04.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:04 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-142"}]: dispatch 2024-02-05T08:06:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:05 smithi157 ceph-mon[33345]: pgmap v1114: 268 pgs: 268 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T08:06:05.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:05 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:05.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:05 smithi157 ceph-mon[33345]: osdmap e717: 8 total, 8 up, 8 in 2024-02-05T08:06:05.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:05 smithi131 ceph-mon[41654]: pgmap v1114: 268 pgs: 268 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T08:06:05.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:05 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:05.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:05 smithi131 ceph-mon[41654]: osdmap e717: 8 total, 8 up, 8 in 2024-02-05T08:06:05.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:05 smithi131 ceph-mon[29507]: pgmap v1114: 268 pgs: 268 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T08:06:05.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:05 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:05.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:05 smithi131 ceph-mon[29507]: osdmap e717: 8 total, 8 up, 8 in 2024-02-05T08:06:06.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:06 smithi157 ceph-mon[33345]: osdmap e718: 8 total, 8 up, 8 in 2024-02-05T08:06:06.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:06 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:06.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:06 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:06.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:06 smithi131 ceph-mon[29507]: osdmap e718: 8 total, 8 up, 8 in 2024-02-05T08:06:06.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:06 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:06.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:06 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:06.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:06 smithi131 ceph-mon[41654]: osdmap e718: 8 total, 8 up, 8 in 2024-02-05T08:06:06.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:06 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:06.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:06 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:07 smithi157 ceph-mon[33345]: pgmap v1117: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T08:06:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:07 smithi157 ceph-mon[33345]: osdmap e719: 8 total, 8 up, 8 in 2024-02-05T08:06:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:07 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:07 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:07.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:07 smithi131 ceph-mon[29507]: pgmap v1117: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T08:06:07.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:07.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:07 smithi131 ceph-mon[29507]: osdmap e719: 8 total, 8 up, 8 in 2024-02-05T08:06:07.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:07 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:07.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:07 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:07.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:07 smithi131 ceph-mon[41654]: pgmap v1117: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T08:06:07.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:07.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:07 smithi131 ceph-mon[41654]: osdmap e719: 8 total, 8 up, 8 in 2024-02-05T08:06:07.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:07 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:07.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:07 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:08.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:06:08.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:08 smithi157 ceph-mon[33345]: osdmap e720: 8 total, 8 up, 8 in 2024-02-05T08:06:08.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:08 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:08.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:08 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:08.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:06:08.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:08 smithi131 ceph-mon[41654]: osdmap e720: 8 total, 8 up, 8 in 2024-02-05T08:06:08.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:08 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:08.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:08 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:08.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:06:08.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:08 smithi131 ceph-mon[29507]: osdmap e720: 8 total, 8 up, 8 in 2024-02-05T08:06:08.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:08 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:08.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:08 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:09.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:06:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:06:09.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:09 smithi157 ceph-mon[33345]: pgmap v1120: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:06:09.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-144"}]': finished 2024-02-05T08:06:09.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:09 smithi157 ceph-mon[33345]: osdmap e721: 8 total, 8 up, 8 in 2024-02-05T08:06:09.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:09 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-144", "mode": "readproxy"}]: dispatch 2024-02-05T08:06:09.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:09 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-144", "mode": "readproxy"}]: dispatch 2024-02-05T08:06:09.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:09 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:09.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:09 smithi131 ceph-mon[41654]: pgmap v1120: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:06:09.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-144"}]': finished 2024-02-05T08:06:09.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:09 smithi131 ceph-mon[41654]: osdmap e721: 8 total, 8 up, 8 in 2024-02-05T08:06:09.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:09 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-144", "mode": "readproxy"}]: dispatch 2024-02-05T08:06:09.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:09 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-144", "mode": "readproxy"}]: dispatch 2024-02-05T08:06:09.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:09 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:09.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:09 smithi131 ceph-mon[29507]: pgmap v1120: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:06:09.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-144"}]': finished 2024-02-05T08:06:09.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:09 smithi131 ceph-mon[29507]: osdmap e721: 8 total, 8 up, 8 in 2024-02-05T08:06:09.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:09 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-144", "mode": "readproxy"}]: dispatch 2024-02-05T08:06:09.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:09 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-144", "mode": "readproxy"}]: dispatch 2024-02-05T08:06:09.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:09 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:10.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:10 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:06:10.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:10 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-144", "mode": "readproxy"}]': finished 2024-02-05T08:06:10.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:10 smithi157 ceph-mon[33345]: osdmap e722: 8 total, 8 up, 8 in 2024-02-05T08:06:10.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:10 smithi157 ceph-mon[33345]: pgmap v1123: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:06:10.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:10 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:06:10.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:10 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-144", "mode": "readproxy"}]': finished 2024-02-05T08:06:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:10 smithi131 ceph-mon[41654]: osdmap e722: 8 total, 8 up, 8 in 2024-02-05T08:06:10.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:10 smithi131 ceph-mon[41654]: pgmap v1123: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:06:10.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:10 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:06:10.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:10 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-144", "mode": "readproxy"}]': finished 2024-02-05T08:06:10.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:10 smithi131 ceph-mon[29507]: osdmap e722: 8 total, 8 up, 8 in 2024-02-05T08:06:10.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:10 smithi131 ceph-mon[29507]: pgmap v1123: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T08:06:12.942 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:12 smithi131 ceph-mon[41654]: pgmap v1124: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 932 B/s rd, 186 B/s wr, 1 op/s 2024-02-05T08:06:12.942 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:12 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:12.942 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:12 smithi131 ceph-mon[29507]: pgmap v1124: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 932 B/s rd, 186 B/s wr, 1 op/s 2024-02-05T08:06:12.942 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:12 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:12.970 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:12 smithi157 ceph-mon[33345]: pgmap v1124: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 932 B/s rd, 186 B/s wr, 1 op/s 2024-02-05T08:06:12.970 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:12 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:06:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:06:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:06:14.970 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:14 smithi157 ceph-mon[33345]: pgmap v1125: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 157 B/s wr, 1 op/s 2024-02-05T08:06:15.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:14 smithi131 ceph-mon[41654]: pgmap v1125: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 157 B/s wr, 1 op/s 2024-02-05T08:06:15.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:14 smithi131 ceph-mon[29507]: pgmap v1125: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 157 B/s wr, 1 op/s 2024-02-05T08:06:16.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:16 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:06:16.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:16.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:16 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:06:16.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:16.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:16 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:06:16.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:17 smithi157 ceph-mon[33345]: pgmap v1126: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-02-05T08:06:17.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:17 smithi131 ceph-mon[41654]: pgmap v1126: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-02-05T08:06:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:17 smithi131 ceph-mon[29507]: pgmap v1126: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-02-05T08:06:19.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:19 smithi157 ceph-mon[33345]: pgmap v1127: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 108 B/s wr, 1 op/s 2024-02-05T08:06:19.471 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:06:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:06:19.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:19 smithi131 ceph-mon[41654]: pgmap v1127: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 108 B/s wr, 1 op/s 2024-02-05T08:06:19.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:19 smithi131 ceph-mon[29507]: pgmap v1127: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 108 B/s wr, 1 op/s 2024-02-05T08:06:20.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:20 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:20.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:20 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:20.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:20 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:20.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:20 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:20.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:20 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:20.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:20 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:21 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:21 smithi157 ceph-mon[33345]: pgmap v1128: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:06:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:06:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:21 smithi157 ceph-mon[33345]: osdmap e723: 8 total, 8 up, 8 in 2024-02-05T08:06:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:21 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:21 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:21 smithi131 ceph-mon[41654]: pgmap v1128: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:06:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:06:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:21 smithi131 ceph-mon[41654]: osdmap e723: 8 total, 8 up, 8 in 2024-02-05T08:06:21.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:21.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:21 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:21.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:21 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:21.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:21 smithi131 ceph-mon[29507]: pgmap v1128: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:06:21.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:06:21.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:21 smithi131 ceph-mon[29507]: osdmap e723: 8 total, 8 up, 8 in 2024-02-05T08:06:21.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:21.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:21 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:22.418 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:22 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:06:22.419 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144"}]': finished 2024-02-05T08:06:22.419 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:22 smithi131 ceph-mon[41654]: osdmap e724: 8 total, 8 up, 8 in 2024-02-05T08:06:22.419 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:22.419 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:22.419 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:22.419 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:22 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:22.419 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:22 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:06:22.419 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144"}]': finished 2024-02-05T08:06:22.419 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:22 smithi131 ceph-mon[29507]: osdmap e724: 8 total, 8 up, 8 in 2024-02-05T08:06:22.419 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:22.419 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:22.420 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:22.420 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:22 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:22.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:22 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:06:22.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144"}]': finished 2024-02-05T08:06:22.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:22 smithi157 ceph-mon[33345]: osdmap e724: 8 total, 8 up, 8 in 2024-02-05T08:06:22.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:22.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:22.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:22.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:22 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-144"}]: dispatch 2024-02-05T08:06:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:06:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:06:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:06:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:23 smithi131 ceph-mon[41654]: pgmap v1131: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:06:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:23 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:23.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:23 smithi131 ceph-mon[41654]: osdmap e725: 8 total, 8 up, 8 in 2024-02-05T08:06:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:23 smithi131 ceph-mon[29507]: pgmap v1131: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:06:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:23 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:23 smithi131 ceph-mon[29507]: osdmap e725: 8 total, 8 up, 8 in 2024-02-05T08:06:23.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:23 smithi157 ceph-mon[33345]: pgmap v1131: 268 pgs: 268 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:06:23.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:23 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:23.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:23 smithi157 ceph-mon[33345]: osdmap e725: 8 total, 8 up, 8 in 2024-02-05T08:06:24.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:24 smithi157 ceph-mon[33345]: osdmap e726: 8 total, 8 up, 8 in 2024-02-05T08:06:24.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:24 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:24.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:24 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:24.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:24 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:06:24.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:24 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:06:24.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:24 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:06:24.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:24 smithi131 ceph-mon[41654]: osdmap e726: 8 total, 8 up, 8 in 2024-02-05T08:06:24.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:24 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:24.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:24 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:24.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:24 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:06:24.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:24 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:06:24.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:24 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:06:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:24 smithi131 ceph-mon[29507]: osdmap e726: 8 total, 8 up, 8 in 2024-02-05T08:06:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:24 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:24 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:24 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:06:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:24 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:06:24.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:24 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:06:25.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:25 smithi157 ceph-mon[33345]: pgmap v1134: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:25.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:25.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:25 smithi157 ceph-mon[33345]: osdmap e727: 8 total, 8 up, 8 in 2024-02-05T08:06:25.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:25 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:25.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:25 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:25.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:25 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:06:25.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:25 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:06:25.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:25 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:06:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:25 smithi131 ceph-mon[41654]: pgmap v1134: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:25 smithi131 ceph-mon[41654]: osdmap e727: 8 total, 8 up, 8 in 2024-02-05T08:06:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:25 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:25 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:25 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:06:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:25 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:06:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:25 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:06:25.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:25 smithi131 ceph-mon[29507]: pgmap v1134: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:25.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:25.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:25 smithi131 ceph-mon[29507]: osdmap e727: 8 total, 8 up, 8 in 2024-02-05T08:06:25.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:25 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:25.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:25 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:25.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:25 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:06:25.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:25 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:06:25.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:25 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:06:26.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:06:26.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:26 smithi157 ceph-mon[33345]: osdmap e728: 8 total, 8 up, 8 in 2024-02-05T08:06:26.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:26 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:26.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:26 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:26.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:06:26.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:26 smithi131 ceph-mon[41654]: osdmap e728: 8 total, 8 up, 8 in 2024-02-05T08:06:26.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:26 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:26.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:26 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:26.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:06:26.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:26 smithi131 ceph-mon[29507]: osdmap e728: 8 total, 8 up, 8 in 2024-02-05T08:06:26.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:26 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:26.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:26 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:27 smithi157 ceph-mon[33345]: pgmap v1137: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-146"}]': finished 2024-02-05T08:06:27.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:27 smithi157 ceph-mon[33345]: osdmap e729: 8 total, 8 up, 8 in 2024-02-05T08:06:27.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:27 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-146", "mode": "writeback"}]: dispatch 2024-02-05T08:06:27.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:27 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-146", "mode": "writeback"}]: dispatch 2024-02-05T08:06:27.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:27 smithi131 ceph-mon[29507]: pgmap v1137: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:27.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-146"}]': finished 2024-02-05T08:06:27.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:27 smithi131 ceph-mon[29507]: osdmap e729: 8 total, 8 up, 8 in 2024-02-05T08:06:27.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:27 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-146", "mode": "writeback"}]: dispatch 2024-02-05T08:06:27.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:27 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-146", "mode": "writeback"}]: dispatch 2024-02-05T08:06:27.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:27 smithi131 ceph-mon[41654]: pgmap v1137: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:27.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-82290-111", "overlaypool": "test-rados-api-smithi131-82290-146"}]': finished 2024-02-05T08:06:27.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:27 smithi131 ceph-mon[41654]: osdmap e729: 8 total, 8 up, 8 in 2024-02-05T08:06:27.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:27 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-146", "mode": "writeback"}]: dispatch 2024-02-05T08:06:27.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:27 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-146", "mode": "writeback"}]: dispatch 2024-02-05T08:06:28.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:28 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:06:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-146", "mode": "writeback"}]': finished 2024-02-05T08:06:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:28 smithi157 ceph-mon[33345]: osdmap e730: 8 total, 8 up, 8 in 2024-02-05T08:06:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:28 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:06:28.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:28 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:06:28.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:28 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:06:28.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-146", "mode": "writeback"}]': finished 2024-02-05T08:06:28.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:28 smithi131 ceph-mon[41654]: osdmap e730: 8 total, 8 up, 8 in 2024-02-05T08:06:28.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:28 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:06:28.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:28 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:06:28.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:28 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T08:06:28.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-82290-146", "mode": "writeback"}]': finished 2024-02-05T08:06:28.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:28 smithi131 ceph-mon[29507]: osdmap e730: 8 total, 8 up, 8 in 2024-02-05T08:06:28.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:28 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:06:28.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:28 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T08:06:29.114 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 08:06:28 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=sqlstore.transactions t=2024-02-05T08:06:28.856488916Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-05T08:06:29.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:06:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:06:29.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:29 smithi157 ceph-mon[33345]: pgmap v1140: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:06:29.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_count","val": "2"}]': finished 2024-02-05T08:06:29.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:29 smithi157 ceph-mon[33345]: osdmap e731: 8 total, 8 up, 8 in 2024-02-05T08:06:29.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:29 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:06:29.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:29 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:06:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:29 smithi131 ceph-mon[41654]: pgmap v1140: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:06:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_count","val": "2"}]': finished 2024-02-05T08:06:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:29 smithi131 ceph-mon[41654]: osdmap e731: 8 total, 8 up, 8 in 2024-02-05T08:06:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:29 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:06:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:29 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:06:29.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:29 smithi131 ceph-mon[29507]: pgmap v1140: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:06:29.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_count","val": "2"}]': finished 2024-02-05T08:06:29.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:29 smithi131 ceph-mon[29507]: osdmap e731: 8 total, 8 up, 8 in 2024-02-05T08:06:29.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:29 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:06:29.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:29 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T08:06:30.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:06:30.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:30 smithi157 ceph-mon[33345]: osdmap e732: 8 total, 8 up, 8 in 2024-02-05T08:06:30.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:06:30.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:30 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:06:30.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:30 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:30.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:06:30.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:30 smithi131 ceph-mon[41654]: osdmap e732: 8 total, 8 up, 8 in 2024-02-05T08:06:30.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:06:30.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:30 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:06:30.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:30 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:30.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_period","val": "600"}]': finished 2024-02-05T08:06:30.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:30 smithi131 ceph-mon[29507]: osdmap e732: 8 total, 8 up, 8 in 2024-02-05T08:06:30.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:06:30.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:30 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T08:06:30.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:30 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:31 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:31 smithi157 ceph-mon[33345]: pgmap v1143: 268 pgs: 268 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-05T08:06:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:31 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:06:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:31 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T08:06:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:31 smithi157 ceph-mon[33345]: osdmap e733: 8 total, 8 up, 8 in 2024-02-05T08:06:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:31 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T08:06:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:31 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T08:06:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:31 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:31 smithi131 ceph-mon[29507]: pgmap v1143: 268 pgs: 268 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-05T08:06:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:31 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:06:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:31 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T08:06:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:31 smithi131 ceph-mon[29507]: osdmap e733: 8 total, 8 up, 8 in 2024-02-05T08:06:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:31 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T08:06:31.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:31 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T08:06:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:31 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:31 smithi131 ceph-mon[41654]: pgmap v1143: 268 pgs: 268 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-05T08:06:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:31 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T08:06:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:31 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T08:06:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:31 smithi131 ceph-mon[41654]: osdmap e733: 8 total, 8 up, 8 in 2024-02-05T08:06:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:31 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T08:06:31.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:31 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T08:06:32.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T08:06:32.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:32 smithi157 ceph-mon[33345]: osdmap e734: 8 total, 8 up, 8 in 2024-02-05T08:06:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:32 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T08:06:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:32 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T08:06:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:32 smithi157 ceph-mon[33345]: pgmap v1146: 268 pgs: 268 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-05T08:06:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:32 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:32.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T08:06:32.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:32 smithi131 ceph-mon[41654]: osdmap e734: 8 total, 8 up, 8 in 2024-02-05T08:06:32.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:32 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T08:06:32.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:32 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T08:06:32.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:32 smithi131 ceph-mon[41654]: pgmap v1146: 268 pgs: 268 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-05T08:06:32.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:32 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T08:06:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:32 smithi131 ceph-mon[29507]: osdmap e734: 8 total, 8 up, 8 in 2024-02-05T08:06:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:32 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T08:06:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:32 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T08:06:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:32 smithi131 ceph-mon[29507]: pgmap v1146: 268 pgs: 268 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-05T08:06:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:32 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:06:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:06:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:06:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:33 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "target_max_objects","val": "1"}]': finished 2024-02-05T08:06:33.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:33 smithi131 ceph-mon[41654]: osdmap e735: 8 total, 8 up, 8 in 2024-02-05T08:06:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:33 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "target_max_objects","val": "1"}]': finished 2024-02-05T08:06:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:33 smithi131 ceph-mon[29507]: osdmap e735: 8 total, 8 up, 8 in 2024-02-05T08:06:33.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:33 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-146","var": "target_max_objects","val": "1"}]': finished 2024-02-05T08:06:33.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:33 smithi157 ceph-mon[33345]: osdmap e735: 8 total, 8 up, 8 in 2024-02-05T08:06:34.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:34 smithi157 ceph-mon[33345]: pgmap v1148: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:06:34.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:34 smithi131 ceph-mon[41654]: pgmap v1148: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:06:34.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:34 smithi131 ceph-mon[29507]: pgmap v1148: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:06:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:35 smithi157 ceph-mon[33345]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-05T08:06:35.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:35 smithi131 ceph-mon[41654]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-05T08:06:35.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:35 smithi131 ceph-mon[29507]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-05T08:06:36.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:36 smithi157 ceph-mon[33345]: pgmap v1149: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:06:36.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:36 smithi131 ceph-mon[41654]: pgmap v1149: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:06:36.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:36 smithi131 ceph-mon[29507]: pgmap v1149: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:06:39.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:38 smithi131 ceph-mon[41654]: pgmap v1150: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 684 B/s rd, 0 B/s wr, 0 op/s 2024-02-05T08:06:39.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:38 smithi131 ceph-mon[29507]: pgmap v1150: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 684 B/s rd, 0 B/s wr, 0 op/s 2024-02-05T08:06:39.137 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:38 smithi157 ceph-mon[33345]: pgmap v1150: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 684 B/s rd, 0 B/s wr, 0 op/s 2024-02-05T08:06:39.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:06:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:06:41.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:40 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:41.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:40 smithi131 ceph-mon[41654]: pgmap v1151: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:06:41.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:40 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:41.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:40 smithi131 ceph-mon[29507]: pgmap v1151: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:06:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:40 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:40 smithi157 ceph-mon[33345]: pgmap v1151: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:06:43.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:42 smithi157 ceph-mon[33345]: pgmap v1152: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:06:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:42 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:43.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:06:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:06:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:06:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:42 smithi131 ceph-mon[29507]: pgmap v1152: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:06:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:42 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:42 smithi131 ceph-mon[41654]: pgmap v1152: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T08:06:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:42 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:43.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:44.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:06:44.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:43 smithi157 ceph-mon[33345]: osdmap e736: 8 total, 8 up, 8 in 2024-02-05T08:06:44.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:43 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:44.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:44.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:06:44.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:43 smithi131 ceph-mon[41654]: osdmap e736: 8 total, 8 up, 8 in 2024-02-05T08:06:44.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:43 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:44.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:44.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:06:44.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:43 smithi131 ceph-mon[29507]: osdmap e736: 8 total, 8 up, 8 in 2024-02-05T08:06:44.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:43 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:44.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:45.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:45 smithi157 ceph-mon[33345]: pgmap v1154: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:06:45.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146"}]': finished 2024-02-05T08:06:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:45 smithi157 ceph-mon[33345]: osdmap e737: 8 total, 8 up, 8 in 2024-02-05T08:06:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:45 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:45.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:45 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:45 smithi131 ceph-mon[41654]: pgmap v1154: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:06:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146"}]': finished 2024-02-05T08:06:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:45 smithi131 ceph-mon[41654]: osdmap e737: 8 total, 8 up, 8 in 2024-02-05T08:06:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:45.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:45 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:45.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:45 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:45 smithi131 ceph-mon[29507]: pgmap v1154: 268 pgs: 268 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:06:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146"}]': finished 2024-02-05T08:06:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:45 smithi131 ceph-mon[29507]: osdmap e737: 8 total, 8 up, 8 in 2024-02-05T08:06:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:45 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:45.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:45 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-146"}]: dispatch 2024-02-05T08:06:46.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:46 smithi157 ceph-mon[33345]: osdmap e738: 8 total, 8 up, 8 in 2024-02-05T08:06:46.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:46 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:06:46.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:46 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:46.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:46 smithi157 ceph-mon[33345]: osdmap e739: 8 total, 8 up, 8 in 2024-02-05T08:06:46.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:46 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:46.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:46 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:46.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:46 smithi131 ceph-mon[41654]: osdmap e738: 8 total, 8 up, 8 in 2024-02-05T08:06:46.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:46 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:06:46.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:46 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:46.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:46 smithi131 ceph-mon[41654]: osdmap e739: 8 total, 8 up, 8 in 2024-02-05T08:06:46.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:46 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:46.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:46 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:46.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:46 smithi131 ceph-mon[29507]: osdmap e738: 8 total, 8 up, 8 in 2024-02-05T08:06:46.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:46 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:06:46.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:46 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:46.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:46 smithi131 ceph-mon[29507]: osdmap e739: 8 total, 8 up, 8 in 2024-02-05T08:06:46.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:46 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:46.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:46 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:47.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:47 smithi157 ceph-mon[33345]: pgmap v1157: 236 pgs: 236 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-05T08:06:47.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:47 smithi157 ceph-mon[33345]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-05T08:06:47.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:47.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:47 smithi157 ceph-mon[33345]: osdmap e740: 8 total, 8 up, 8 in 2024-02-05T08:06:47.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:47 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:47.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:47 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:47.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:47 smithi131 ceph-mon[41654]: pgmap v1157: 236 pgs: 236 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-05T08:06:47.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:47 smithi131 ceph-mon[41654]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-05T08:06:47.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:47.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:47 smithi131 ceph-mon[41654]: osdmap e740: 8 total, 8 up, 8 in 2024-02-05T08:06:47.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:47 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:47.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:47 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:47.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:47 smithi131 ceph-mon[29507]: pgmap v1157: 236 pgs: 236 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-05T08:06:47.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:47 smithi131 ceph-mon[29507]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-05T08:06:47.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:47.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:47 smithi131 ceph-mon[29507]: osdmap e740: 8 total, 8 up, 8 in 2024-02-05T08:06:47.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:47 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:47.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:47 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T08:06:49.143 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:48 smithi157 ceph-mon[33345]: pgmap v1160: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:06:49.143 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:06:49.143 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:48 smithi157 ceph-mon[33345]: osdmap e741: 8 total, 8 up, 8 in 2024-02-05T08:06:49.143 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:48 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148"}]: dispatch 2024-02-05T08:06:49.143 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:48 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148"}]: dispatch 2024-02-05T08:06:49.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:48 smithi131 ceph-mon[41654]: pgmap v1160: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:06:49.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:06:49.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:48 smithi131 ceph-mon[41654]: osdmap e741: 8 total, 8 up, 8 in 2024-02-05T08:06:49.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:48 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148"}]: dispatch 2024-02-05T08:06:49.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:48 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148"}]: dispatch 2024-02-05T08:06:49.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:48 smithi131 ceph-mon[29507]: pgmap v1160: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:06:49.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T08:06:49.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:48 smithi131 ceph-mon[29507]: osdmap e741: 8 total, 8 up, 8 in 2024-02-05T08:06:49.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:48 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148"}]: dispatch 2024-02-05T08:06:49.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:48 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148"}]: dispatch 2024-02-05T08:06:49.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:06:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:06:50.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148"}]': finished 2024-02-05T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:49 smithi157 ceph-mon[33345]: osdmap e742: 8 total, 8 up, 8 in 2024-02-05T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:49 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148"}]: dispatch 2024-02-05T08:06:50.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:49 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148"}]: dispatch 2024-02-05T08:06:50.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148"}]': finished 2024-02-05T08:06:50.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:49 smithi131 ceph-mon[41654]: osdmap e742: 8 total, 8 up, 8 in 2024-02-05T08:06:50.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:50.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:50.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:49 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148"}]: dispatch 2024-02-05T08:06:50.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:49 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148"}]: dispatch 2024-02-05T08:06:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148"}]': finished 2024-02-05T08:06:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:49 smithi131 ceph-mon[29507]: osdmap e742: 8 total, 8 up, 8 in 2024-02-05T08:06:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:49 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148"}]: dispatch 2024-02-05T08:06:50.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:49 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-148"}]: dispatch 2024-02-05T08:06:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:50 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:50 smithi157 ceph-mon[33345]: pgmap v1163: 268 pgs: 268 active+clean; 4.4 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T08:06:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:50 smithi157 ceph-mon[33345]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:51.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:50 smithi157 ceph-mon[33345]: osdmap e743: 8 total, 8 up, 8 in 2024-02-05T08:06:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:50 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:50 smithi131 ceph-mon[41654]: pgmap v1163: 268 pgs: 268 active+clean; 4.4 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T08:06:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:50 smithi131 ceph-mon[41654]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:50 smithi131 ceph-mon[41654]: osdmap e743: 8 total, 8 up, 8 in 2024-02-05T08:06:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:50 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:50 smithi131 ceph-mon[29507]: pgmap v1163: 268 pgs: 268 active+clean; 4.4 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T08:06:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:50 smithi131 ceph-mon[29507]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:51.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:50 smithi131 ceph-mon[29507]: osdmap e743: 8 total, 8 up, 8 in 2024-02-05T08:06:52.168 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:51 smithi131 ceph-mon[41654]: osdmap e744: 8 total, 8 up, 8 in 2024-02-05T08:06:52.168 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:51 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:52.168 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:51 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:52.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:51 smithi131 ceph-mon[29507]: osdmap e744: 8 total, 8 up, 8 in 2024-02-05T08:06:52.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:51 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:52.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:51 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:52.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:51 smithi157 ceph-mon[33345]: osdmap e744: 8 total, 8 up, 8 in 2024-02-05T08:06:52.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:51 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:52.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:51 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:53.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:52 smithi157 ceph-mon[33345]: pgmap v1166: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T08:06:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:52 smithi157 ceph-mon[33345]: osdmap e745: 8 total, 8 up, 8 in 2024-02-05T08:06:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-150"}]: dispatch 2024-02-05T08:06:53.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-150"}]: dispatch 2024-02-05T08:06:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:06:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:06:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:06:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:52 smithi131 ceph-mon[41654]: pgmap v1166: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T08:06:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:52 smithi131 ceph-mon[41654]: osdmap e745: 8 total, 8 up, 8 in 2024-02-05T08:06:53.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:53.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:53.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-150"}]: dispatch 2024-02-05T08:06:53.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-150"}]: dispatch 2024-02-05T08:06:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:52 smithi131 ceph-mon[29507]: pgmap v1166: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T08:06:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:52 smithi131 ceph-mon[29507]: osdmap e745: 8 total, 8 up, 8 in 2024-02-05T08:06:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-150"}]: dispatch 2024-02-05T08:06:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-150"}]: dispatch 2024-02-05T08:06:54.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:53 smithi157 ceph-mon[33345]: osdmap e746: 8 total, 8 up, 8 in 2024-02-05T08:06:54.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:53 smithi131 ceph-mon[41654]: osdmap e746: 8 total, 8 up, 8 in 2024-02-05T08:06:54.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:53 smithi131 ceph-mon[29507]: osdmap e746: 8 total, 8 up, 8 in 2024-02-05T08:06:55.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:54 smithi157 ceph-mon[33345]: pgmap v1169: 236 pgs: 236 active+clean; 4.4 MiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T08:06:55.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:54 smithi157 ceph-mon[33345]: osdmap e747: 8 total, 8 up, 8 in 2024-02-05T08:06:55.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:55.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:54 smithi131 ceph-mon[41654]: pgmap v1169: 236 pgs: 236 active+clean; 4.4 MiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T08:06:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:54 smithi131 ceph-mon[41654]: osdmap e747: 8 total, 8 up, 8 in 2024-02-05T08:06:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:55.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:54 smithi131 ceph-mon[29507]: pgmap v1169: 236 pgs: 236 active+clean; 4.4 MiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T08:06:55.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:54 smithi131 ceph-mon[29507]: osdmap e747: 8 total, 8 up, 8 in 2024-02-05T08:06:55.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:55.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:56.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:56.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:55 smithi157 ceph-mon[33345]: osdmap e748: 8 total, 8 up, 8 in 2024-02-05T08:06:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-152"}]: dispatch 2024-02-05T08:06:56.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:55 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-152"}]: dispatch 2024-02-05T08:06:56.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:56.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:55 smithi131 ceph-mon[41654]: osdmap e748: 8 total, 8 up, 8 in 2024-02-05T08:06:56.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:56.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:56.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-152"}]: dispatch 2024-02-05T08:06:56.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:55 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-152"}]: dispatch 2024-02-05T08:06:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:55 smithi131 ceph-mon[29507]: osdmap e748: 8 total, 8 up, 8 in 2024-02-05T08:06:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-152"}]: dispatch 2024-02-05T08:06:56.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:55 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-152"}]: dispatch 2024-02-05T08:06:57.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:56 smithi157 ceph-mon[33345]: pgmap v1172: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T08:06:57.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:56 smithi157 ceph-mon[33345]: osdmap e749: 8 total, 8 up, 8 in 2024-02-05T08:06:57.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:56 smithi157 ceph-mon[33345]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:56 smithi131 ceph-mon[41654]: pgmap v1172: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T08:06:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:56 smithi131 ceph-mon[41654]: osdmap e749: 8 total, 8 up, 8 in 2024-02-05T08:06:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:56 smithi131 ceph-mon[41654]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:56 smithi131 ceph-mon[29507]: pgmap v1172: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T08:06:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:56 smithi131 ceph-mon[29507]: osdmap e749: 8 total, 8 up, 8 in 2024-02-05T08:06:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:56 smithi131 ceph-mon[29507]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:57.765 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.765 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.765 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.765 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.765 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.765 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.766 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.766 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.766 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.766 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.766 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.766 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.766 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.766 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.766 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.766 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.766 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.766 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.766 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.766 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.767 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.767 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.767 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.767 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.767 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.767 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.767 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.767 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.767 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.767 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.767 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.767 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.768 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.768 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.768 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.768 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.768 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.768 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.768 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.768 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.768 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.768 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.768 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.768 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.768 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.768 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.769 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.769 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.769 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.769 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.769 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.769 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.769 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.769 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.769 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.769 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.769 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.769 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.769 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.769 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.770 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.770 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.770 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.770 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.770 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.770 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.770 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.770 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.770 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.770 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.770 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.770 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.770 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.770 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.771 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:57.772 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:58.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:57 smithi157 ceph-mon[33345]: osdmap e750: 8 total, 8 up, 8 in 2024-02-05T08:06:58.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:57 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:58.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:57 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:58.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:57 smithi131 ceph-mon[41654]: osdmap e750: 8 total, 8 up, 8 in 2024-02-05T08:06:58.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:57 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:58.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:57 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:58.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:57 smithi131 ceph-mon[29507]: osdmap e750: 8 total, 8 up, 8 in 2024-02-05T08:06:58.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:57 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:58.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:57 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:59.154 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:58 smithi157 ceph-mon[33345]: pgmap v1175: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 832 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:06:59.154 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:59.154 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:58 smithi157 ceph-mon[33345]: osdmap e751: 8 total, 8 up, 8 in 2024-02-05T08:06:59.154 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-111","var": "dedup_tier","val": "test-rados-api-smithi131-82290-154"}]: dispatch 2024-02-05T08:06:59.154 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-111","var": "dedup_tier","val": "test-rados-api-smithi131-82290-154"}]: dispatch 2024-02-05T08:06:59.154 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:59.154 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:59.155 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-154"}]: dispatch 2024-02-05T08:06:59.155 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-154"}]: dispatch 2024-02-05T08:06:59.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[41654]: pgmap v1175: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 832 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:06:59.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:59.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[41654]: osdmap e751: 8 total, 8 up, 8 in 2024-02-05T08:06:59.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-111","var": "dedup_tier","val": "test-rados-api-smithi131-82290-154"}]: dispatch 2024-02-05T08:06:59.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-111","var": "dedup_tier","val": "test-rados-api-smithi131-82290-154"}]: dispatch 2024-02-05T08:06:59.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-154"}]: dispatch 2024-02-05T08:06:59.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-154"}]: dispatch 2024-02-05T08:06:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[29507]: pgmap v1175: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 832 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:06:59.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-82290-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:06:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[29507]: osdmap e751: 8 total, 8 up, 8 in 2024-02-05T08:06:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-111","var": "dedup_tier","val": "test-rados-api-smithi131-82290-154"}]: dispatch 2024-02-05T08:06:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-82290-111","var": "dedup_tier","val": "test-rados-api-smithi131-82290-154"}]: dispatch 2024-02-05T08:06:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:06:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-154"}]: dispatch 2024-02-05T08:06:59.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-82290-111", "tierpool": "test-rados-api-smithi131-82290-154"}]: dispatch 2024-02-05T08:06:59.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:06:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:07:00.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:06:59 smithi157 ceph-mon[33345]: osdmap e752: 8 total, 8 up, 8 in 2024-02-05T08:07:00.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:06:59 smithi131 ceph-mon[41654]: osdmap e752: 8 total, 8 up, 8 in 2024-02-05T08:07:00.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:06:59 smithi131 ceph-mon[29507]: osdmap e752: 8 total, 8 up, 8 in 2024-02-05T08:07:01.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:00 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:00 smithi157 ceph-mon[33345]: pgmap v1178: 236 pgs: 236 active+clean; 4.4 MiB data, 854 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-05T08:07:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:00 smithi157 ceph-mon[33345]: osdmap e753: 8 total, 8 up, 8 in 2024-02-05T08:07:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:00 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:07:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:07:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:00 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:07:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:00 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:00 smithi131 ceph-mon[41654]: pgmap v1178: 236 pgs: 236 active+clean; 4.4 MiB data, 854 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-05T08:07:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:00 smithi131 ceph-mon[41654]: osdmap e753: 8 total, 8 up, 8 in 2024-02-05T08:07:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:00 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:07:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:07:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:00 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:07:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:00 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:00 smithi131 ceph-mon[29507]: pgmap v1178: 236 pgs: 236 active+clean; 4.4 MiB data, 854 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-05T08:07:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:00 smithi131 ceph-mon[29507]: osdmap e753: 8 total, 8 up, 8 in 2024-02-05T08:07:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:00 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:07:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:07:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:00 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:07:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:01.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: wait LibRadosTwoPoolsECPP.TryFlush (7852 ms) 2024-02-05T08:07:01.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-02-05T08:07:01.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (13064 ms) 2024-02-05T08:07:01.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-02-05T08:07:01.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8047 ms) 2024-02-05T08:07:01.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-02-05T08:07:01.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13120 ms) 2024-02-05T08:07:01.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-02-05T08:07:01.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7789 ms) 2024-02-05T08:07:01.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-02-05T08:07:01.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8057 ms) 2024-02-05T08:07:01.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-02-05T08:07:01.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-05T08:07:01.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: ok, hit_set contains 336:602f83fe:::foo:head 2024-02-05T08:07:01.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9047 ms) 2024-02-05T08:07:01.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-02-05T08:07:01.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707120336,0 2024-02-05T08:07:01.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: first is 1707120336 2024-02-05T08:07:01.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707120336,0 2024-02-05T08:07:01.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707120336,0 2024-02-05T08:07:01.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707120336,1707120338,1707120339,0 2024-02-05T08:07:01.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707120336,1707120338,1707120339,0 2024-02-05T08:07:01.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707120336,1707120338,1707120339,0 2024-02-05T08:07:01.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707120336,1707120338,1707120339,1707120341,1707120342,0 2024-02-05T08:07:01.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707120336,1707120338,1707120339,1707120341,1707120342,0 2024-02-05T08:07:01.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707120336,1707120338,1707120339,1707120341,1707120342,0 2024-02-05T08:07:01.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707120336,1707120338,1707120339,1707120341,1707120342,1707120344,1707120345,0 2024-02-05T08:07:01.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707120336,1707120338,1707120339,1707120341,1707120342,1707120344,1707120345,0 2024-02-05T08:07:01.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707120336,1707120338,1707120339,1707120341,1707120342,1707120344,1707120345,0 2024-02-05T08:07:01.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1707120339,1707120341,1707120342,1707120344,1707120345,1707120347,1707120348,0 2024-02-05T08:07:01.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: first now 1707120339, trimmed 2024-02-05T08:07:01.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (21101 ms) 2024-02-05T08:07:01.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-02-05T08:07:01.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: foo0 2024-02-05T08:07:01.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-05T08:07:01.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14113 ms) 2024-02-05T08:07:01.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-02-05T08:07:01.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (17975 ms) 2024-02-05T08:07:01.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-02-05T08:07:01.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22717 ms) 2024-02-05T08:07:01.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-02-05T08:07:01.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5024 ms) 2024-02-05T08:07:01.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-02-05T08:07:01.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3022 ms) 2024-02-05T08:07:01.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-02-05T08:07:01.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3025 ms) 2024-02-05T08:07:01.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-02-05T08:07:01.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3021 ms) 2024-02-05T08:07:01.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (232742 ms total) 2024-02-05T08:07:01.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: 2024-02-05T08:07:01.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-02-05T08:07:01.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (863091 ms total) 2024-02-05T08:07:01.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-02-05T08:07:01.907 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:01.907 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82008 2024-02-05T08:07:01.907 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82008 2024-02-05T08:07:01.907 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:01.907 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82232 2024-02-05T08:07:01.907 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82232 2024-02-05T08:07:01.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:01.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82745 2024-02-05T08:07:01.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82745 2024-02-05T08:07:01.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:01.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82468 2024-02-05T08:07:01.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82468 2024-02-05T08:07:01.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:01.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82346 2024-02-05T08:07:01.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82346 2024-02-05T08:07:01.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:01.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82090 2024-02-05T08:07:01.908 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82090 2024-02-05T08:07:02.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:07:02.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:01 smithi157 ceph-mon[33345]: osdmap e754: 8 total, 8 up, 8 in 2024-02-05T08:07:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:01 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:07:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:07:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:01 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:07:02.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:01 smithi157 ceph-mon[33345]: osdmap e755: 8 total, 8 up, 8 in 2024-02-05T08:07:02.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:07:02.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:01 smithi131 ceph-mon[41654]: osdmap e754: 8 total, 8 up, 8 in 2024-02-05T08:07:02.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:01 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:07:02.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:07:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:01 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:07:02.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:01 smithi131 ceph-mon[41654]: osdmap e755: 8 total, 8 up, 8 in 2024-02-05T08:07:02.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:07:02.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:01 smithi131 ceph-mon[29507]: osdmap e754: 8 total, 8 up, 8 in 2024-02-05T08:07:02.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:01 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2731098802' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:07:02.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-111"}]: dispatch 2024-02-05T08:07:02.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:01 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-82290-111"}]': finished 2024-02-05T08:07:02.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:01 smithi131 ceph-mon[29507]: osdmap e755: 8 total, 8 up, 8 in 2024-02-05T08:07:03.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:02 smithi157 ceph-mon[33345]: pgmap v1181: 228 pgs: 228 active+clean; 582 KiB data, 854 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:07:03.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:02 smithi157 ceph-mon[33345]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:07:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:07:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:07:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:07:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:02 smithi131 ceph-mon[41654]: pgmap v1181: 228 pgs: 228 active+clean; 582 KiB data, 854 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:07:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:02 smithi131 ceph-mon[41654]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:07:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:02 smithi131 ceph-mon[29507]: pgmap v1181: 228 pgs: 228 active+clean; 582 KiB data, 854 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:07:03.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:02 smithi131 ceph-mon[29507]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:07:05.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:04 smithi157 ceph-mon[33345]: pgmap v1183: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:07:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:04 smithi131 ceph-mon[41654]: pgmap v1183: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:07:05.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:04 smithi131 ceph-mon[29507]: pgmap v1183: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:07:07.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:06 smithi157 ceph-mon[33345]: pgmap v1184: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:07:07.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:06 smithi131 ceph-mon[41654]: pgmap v1184: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:07:07.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:06 smithi131 ceph-mon[29507]: pgmap v1184: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:07:09.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:08 smithi157 ceph-mon[33345]: pgmap v1185: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 662 B/s rd, 0 op/s 2024-02-05T08:07:09.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:07:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:07:09.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:08 smithi131 ceph-mon[41654]: pgmap v1185: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 662 B/s rd, 0 op/s 2024-02-05T08:07:09.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:08 smithi131 ceph-mon[29507]: pgmap v1185: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 662 B/s rd, 0 op/s 2024-02-05T08:07:11.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:10 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:11.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:10 smithi157 ceph-mon[33345]: pgmap v1186: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:07:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:10 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:10 smithi131 ceph-mon[41654]: pgmap v1186: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:07:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:10 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:10 smithi131 ceph-mon[29507]: pgmap v1186: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:07:13.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:12 smithi157 ceph-mon[33345]: pgmap v1187: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:07:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:07:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:07:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:07:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:12 smithi131 ceph-mon[41654]: pgmap v1187: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:07:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:12 smithi131 ceph-mon[29507]: pgmap v1187: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:07:15.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:14 smithi157 ceph-mon[33345]: pgmap v1188: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:07:15.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:14 smithi131 ceph-mon[41654]: pgmap v1188: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:07:15.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:14 smithi131 ceph-mon[29507]: pgmap v1188: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:07:16.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:16.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:16.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:17.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:16 smithi157 ceph-mon[33345]: pgmap v1189: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:17.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:16 smithi131 ceph-mon[41654]: pgmap v1189: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:16 smithi131 ceph-mon[29507]: pgmap v1189: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:19.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:18 smithi157 ceph-mon[33345]: pgmap v1190: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:19.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:07:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:07:19.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:18 smithi131 ceph-mon[41654]: pgmap v1190: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:19.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:18 smithi131 ceph-mon[29507]: pgmap v1190: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:21.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:20 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:21.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:20 smithi157 ceph-mon[33345]: pgmap v1191: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:21.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:20 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:21.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:20 smithi131 ceph-mon[41654]: pgmap v1191: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:20 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:20 smithi131 ceph-mon[29507]: pgmap v1191: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:23.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:22 smithi157 ceph-mon[33345]: pgmap v1192: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:22 smithi131 ceph-mon[41654]: pgmap v1192: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:23.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:22 smithi131 ceph-mon[29507]: pgmap v1192: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:23.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:07:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:07:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:07:25.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:24 smithi157 ceph-mon[33345]: pgmap v1193: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:25.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:24 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:07:25.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:24 smithi131 ceph-mon[41654]: pgmap v1193: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:25.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:24 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:07:25.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:24 smithi131 ceph-mon[29507]: pgmap v1193: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:25.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:24 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:07:26.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:25 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:07:26.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:25 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:07:26.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:25 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:07:26.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:25 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:07:26.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:25 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:07:26.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:25 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:07:26.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:25 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:07:26.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:25 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:07:26.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:25 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:07:27.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:26 smithi157 ceph-mon[33345]: pgmap v1194: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:26 smithi131 ceph-mon[41654]: pgmap v1194: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:27.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:26 smithi131 ceph-mon[29507]: pgmap v1194: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:29.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:28 smithi157 ceph-mon[33345]: pgmap v1195: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:29.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:07:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:07:29.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:28 smithi131 ceph-mon[41654]: pgmap v1195: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:29.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:28 smithi131 ceph-mon[29507]: pgmap v1195: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:30 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:30 smithi157 ceph-mon[33345]: pgmap v1196: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:31.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:30 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:30 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:30 smithi131 ceph-mon[41654]: pgmap v1196: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:30 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:30 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:30 smithi131 ceph-mon[29507]: pgmap v1196: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:31.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:30 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:33.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:32 smithi157 ceph-mon[33345]: pgmap v1197: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:32 smithi131 ceph-mon[41654]: pgmap v1197: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:33.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:32 smithi131 ceph-mon[29507]: pgmap v1197: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:07:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:07:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:07:35.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:34 smithi157 ceph-mon[33345]: pgmap v1198: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:34 smithi131 ceph-mon[41654]: pgmap v1198: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:35.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:34 smithi131 ceph-mon[29507]: pgmap v1198: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:37.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:36 smithi157 ceph-mon[33345]: pgmap v1199: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:37.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:36 smithi131 ceph-mon[41654]: pgmap v1199: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:37.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:36 smithi131 ceph-mon[29507]: pgmap v1199: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:39.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:38 smithi157 ceph-mon[33345]: pgmap v1200: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:39.221 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:07:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:07:39.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:38 smithi131 ceph-mon[41654]: pgmap v1200: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:39.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:38 smithi131 ceph-mon[29507]: pgmap v1200: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:40 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:40 smithi131 ceph-mon[41654]: pgmap v1201: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:40 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:40 smithi131 ceph-mon[29507]: pgmap v1201: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:40 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:40 smithi157 ceph-mon[33345]: pgmap v1201: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:42 smithi131 ceph-mon[41654]: pgmap v1202: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:43.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:42 smithi131 ceph-mon[29507]: pgmap v1202: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:43.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:07:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:07:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:07:43.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:42 smithi157 ceph-mon[33345]: pgmap v1202: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:45.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:44 smithi131 ceph-mon[41654]: pgmap v1203: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:45.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:44 smithi131 ceph-mon[29507]: pgmap v1203: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:45.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:44 smithi157 ceph-mon[33345]: pgmap v1203: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:45 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:46.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:45 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:46.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:45 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:47.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:46 smithi131 ceph-mon[41654]: pgmap v1204: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:47.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:46 smithi131 ceph-mon[29507]: pgmap v1204: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:47.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:46 smithi157 ceph-mon[33345]: pgmap v1204: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:49.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:48 smithi131 ceph-mon[41654]: pgmap v1205: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:49.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:48 smithi131 ceph-mon[29507]: pgmap v1205: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:49.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:48 smithi157 ceph-mon[33345]: pgmap v1205: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:49.471 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:07:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:07:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:50 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:50 smithi131 ceph-mon[41654]: pgmap v1206: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:50 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:50 smithi131 ceph-mon[29507]: pgmap v1206: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:50 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:50 smithi157 ceph-mon[33345]: pgmap v1206: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:52 smithi131 ceph-mon[41654]: pgmap v1207: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:53.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:52 smithi131 ceph-mon[29507]: pgmap v1207: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:53.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:07:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:07:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:07:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:52 smithi157 ceph-mon[33345]: pgmap v1207: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:55.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:54 smithi131 ceph-mon[29507]: pgmap v1208: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:55 smithi131 ceph-mon[41654]: pgmap v1208: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:54 smithi157 ceph-mon[33345]: pgmap v1208: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:57 smithi131 ceph-mon[29507]: pgmap v1209: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:57 smithi131 ceph-mon[41654]: pgmap v1209: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:57 smithi157 ceph-mon[33345]: pgmap v1209: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:07:59.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:07:59 smithi131 ceph-mon[29507]: pgmap v1210: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:59.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:07:59 smithi131 ceph-mon[41654]: pgmap v1210: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:59.273 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:07:59 smithi157 ceph-mon[33345]: pgmap v1210: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:59.273 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:07:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:08:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:01 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:01 smithi131 ceph-mon[29507]: pgmap v1211: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:01 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:01 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:01 smithi131 ceph-mon[41654]: pgmap v1211: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:01.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:01 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:01 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:01 smithi157 ceph-mon[33345]: pgmap v1211: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:01 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:03 smithi131 ceph-mon[41654]: pgmap v1212: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:03 smithi131 ceph-mon[29507]: pgmap v1212: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:08:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:08:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:08:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:03 smithi157 ceph-mon[33345]: pgmap v1212: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:05 smithi157 ceph-mon[33345]: pgmap v1213: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:05.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:05 smithi131 ceph-mon[41654]: pgmap v1213: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:05.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:05 smithi131 ceph-mon[29507]: pgmap v1213: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:07.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:07 smithi157 ceph-mon[33345]: pgmap v1214: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:07.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:07 smithi131 ceph-mon[41654]: pgmap v1214: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:07.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:07 smithi131 ceph-mon[29507]: pgmap v1214: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:09.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:09 smithi157 ceph-mon[33345]: pgmap v1215: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:09.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:08:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:08:09.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:09 smithi131 ceph-mon[41654]: pgmap v1215: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:09.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:09 smithi131 ceph-mon[29507]: pgmap v1215: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:11 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:11 smithi157 ceph-mon[33345]: pgmap v1216: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:11 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:11 smithi131 ceph-mon[41654]: pgmap v1216: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:11 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:11 smithi131 ceph-mon[29507]: pgmap v1216: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:13 smithi131 ceph-mon[41654]: pgmap v1217: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:13 smithi131 ceph-mon[29507]: pgmap v1217: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:13.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:08:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:08:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:08:13.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:13 smithi157 ceph-mon[33345]: pgmap v1217: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:15 smithi157 ceph-mon[33345]: pgmap v1218: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:15.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:15 smithi131 ceph-mon[41654]: pgmap v1218: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:15.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:15 smithi131 ceph-mon[29507]: pgmap v1218: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:16.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:16.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:16.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:17 smithi157 ceph-mon[33345]: pgmap v1219: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:17.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:17 smithi131 ceph-mon[41654]: pgmap v1219: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:17 smithi131 ceph-mon[29507]: pgmap v1219: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:19.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:19 smithi157 ceph-mon[33345]: pgmap v1220: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:19.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:08:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:08:19.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:19 smithi131 ceph-mon[41654]: pgmap v1220: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:19.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:19 smithi131 ceph-mon[29507]: pgmap v1220: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:21 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:21.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:21 smithi157 ceph-mon[33345]: pgmap v1221: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:21 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:21 smithi131 ceph-mon[41654]: pgmap v1221: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:21.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:21 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:21.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:21 smithi131 ceph-mon[29507]: pgmap v1221: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:23 smithi131 ceph-mon[41654]: pgmap v1222: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:23.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:23 smithi131 ceph-mon[29507]: pgmap v1222: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:23.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:08:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:08:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:08:23.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:23 smithi157 ceph-mon[33345]: pgmap v1222: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:25.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:25 smithi157 ceph-mon[33345]: pgmap v1223: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:25.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:25 smithi131 ceph-mon[29507]: pgmap v1223: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:25 smithi131 ceph-mon[41654]: pgmap v1223: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:26.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:26 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:08:26.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:26 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:08:26.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:26 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:08:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:27 smithi157 ceph-mon[33345]: pgmap v1224: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:08:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:08:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:27 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:08:27.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:27 smithi131 ceph-mon[41654]: pgmap v1224: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:27.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:08:27.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:08:27.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:27 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:08:27.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:27 smithi131 ceph-mon[29507]: pgmap v1224: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:27.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:08:27.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:08:27.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:27 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:08:29.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:29 smithi157 ceph-mon[33345]: pgmap v1225: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:29.471 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:08:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:08:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:29 smithi131 ceph-mon[41654]: pgmap v1225: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:29 smithi131 ceph-mon[29507]: pgmap v1225: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:31 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:31 smithi157 ceph-mon[33345]: pgmap v1226: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:31.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:31 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:31 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:31 smithi131 ceph-mon[41654]: pgmap v1226: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:31 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:31 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:31.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:31 smithi131 ceph-mon[29507]: pgmap v1226: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:31.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:31 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:08:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:08:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:08:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:33 smithi131 ceph-mon[41654]: pgmap v1227: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:33 smithi131 ceph-mon[29507]: pgmap v1227: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:33.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:33 smithi157 ceph-mon[33345]: pgmap v1227: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:35 smithi157 ceph-mon[33345]: pgmap v1228: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:35.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:35 smithi131 ceph-mon[41654]: pgmap v1228: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:35.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:35 smithi131 ceph-mon[29507]: pgmap v1228: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:37 smithi157 ceph-mon[33345]: pgmap v1229: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:37.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:37 smithi131 ceph-mon[41654]: pgmap v1229: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:37.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:37 smithi131 ceph-mon[29507]: pgmap v1229: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:37.776 INFO:tasks.workunit.client.0.smithi131.stdout:ing 1 sec 2024-02-05T08:08:37.776 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.776 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.776 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.776 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.776 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.777 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.777 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.777 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.777 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.777 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.777 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.777 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.777 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.777 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.777 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.777 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.777 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.777 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.777 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.777 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.778 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.778 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.778 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.778 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.778 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.778 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.778 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.778 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.778 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.778 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.778 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.778 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.778 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.778 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.778 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.778 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.779 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.779 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.779 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.779 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.779 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.779 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.779 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.779 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.779 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.779 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.779 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.779 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.779 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.779 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.779 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.780 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.781 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.782 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:37.783 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:08:39.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:39 smithi157 ceph-mon[33345]: pgmap v1230: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:39.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:08:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:08:39.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:39 smithi131 ceph-mon[41654]: pgmap v1230: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:39.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:39 smithi131 ceph-mon[29507]: pgmap v1230: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:41 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:41.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:41 smithi157 ceph-mon[33345]: pgmap v1231: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:41 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:41 smithi131 ceph-mon[29507]: pgmap v1231: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:41 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:41 smithi131 ceph-mon[41654]: pgmap v1231: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:43 smithi131 ceph-mon[41654]: pgmap v1232: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:43 smithi131 ceph-mon[29507]: pgmap v1232: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:43.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:08:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:08:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:08:43.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:43 smithi157 ceph-mon[33345]: pgmap v1232: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:45.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:45 smithi157 ceph-mon[33345]: pgmap v1233: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:45 smithi131 ceph-mon[41654]: pgmap v1233: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:45.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:45 smithi131 ceph-mon[29507]: pgmap v1233: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:46.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:46 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:46.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:46 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:46.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:46 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:47.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:47 smithi157 ceph-mon[33345]: pgmap v1234: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:47.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:47 smithi131 ceph-mon[41654]: pgmap v1234: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:47.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:47 smithi131 ceph-mon[29507]: pgmap v1234: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:49.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:49 smithi157 ceph-mon[33345]: pgmap v1235: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:49.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:08:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:08:49.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:49 smithi131 ceph-mon[41654]: pgmap v1235: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:49.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:49 smithi131 ceph-mon[29507]: pgmap v1235: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:51 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:51 smithi157 ceph-mon[33345]: pgmap v1236: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:51.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:51 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:51.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:51 smithi131 ceph-mon[41654]: pgmap v1236: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:51 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:51 smithi131 ceph-mon[29507]: pgmap v1236: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:08:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:08:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:08:53.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:53 smithi131 ceph-mon[41654]: pgmap v1237: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:53 smithi131 ceph-mon[29507]: pgmap v1237: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:53 smithi157 ceph-mon[33345]: pgmap v1237: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:55 smithi157 ceph-mon[33345]: pgmap v1238: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:55 smithi131 ceph-mon[41654]: pgmap v1238: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:55.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:55 smithi131 ceph-mon[29507]: pgmap v1238: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:57 smithi157 ceph-mon[33345]: pgmap v1239: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:57 smithi131 ceph-mon[41654]: pgmap v1239: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:57 smithi131 ceph-mon[29507]: pgmap v1239: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:08:59.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:08:59 smithi157 ceph-mon[33345]: pgmap v1240: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:59.471 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:08:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:08:59.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:08:59 smithi131 ceph-mon[41654]: pgmap v1240: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:59.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:08:59 smithi131 ceph-mon[29507]: pgmap v1240: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:01 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:01 smithi157 ceph-mon[33345]: pgmap v1241: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:01 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:01 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:01 smithi131 ceph-mon[41654]: pgmap v1241: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:01 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:01 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:01 smithi131 ceph-mon[29507]: pgmap v1241: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:01 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:03 smithi131 ceph-mon[41654]: pgmap v1242: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:03 smithi131 ceph-mon[29507]: pgmap v1242: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:09:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:09:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:09:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:03 smithi157 ceph-mon[33345]: pgmap v1242: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:05 smithi157 ceph-mon[33345]: pgmap v1243: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:05.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:05 smithi131 ceph-mon[41654]: pgmap v1243: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:05.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:05 smithi131 ceph-mon[29507]: pgmap v1243: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:07 smithi157 ceph-mon[33345]: pgmap v1244: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:07.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:07 smithi131 ceph-mon[29507]: pgmap v1244: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:07.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:07 smithi131 ceph-mon[41654]: pgmap v1244: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:09.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:09 smithi157 ceph-mon[33345]: pgmap v1245: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:09.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:09:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:09:09.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:09 smithi131 ceph-mon[41654]: pgmap v1245: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:09.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:09 smithi131 ceph-mon[29507]: pgmap v1245: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:10.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:10 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:10.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:10 smithi157 ceph-mon[33345]: pgmap v1246: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:10.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:10 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:10.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:10 smithi131 ceph-mon[41654]: pgmap v1246: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:10.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:10 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:10.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:10 smithi131 ceph-mon[29507]: pgmap v1246: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:13.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:12 smithi157 ceph-mon[33345]: pgmap v1247: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:12 smithi131 ceph-mon[41654]: pgmap v1247: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:12 smithi131 ceph-mon[29507]: pgmap v1247: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:09:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:09:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:09:15.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:14 smithi157 ceph-mon[33345]: pgmap v1248: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:15.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:14 smithi131 ceph-mon[41654]: pgmap v1248: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:15.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:14 smithi131 ceph-mon[29507]: pgmap v1248: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:16.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:16.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:16.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:17.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:16 smithi157 ceph-mon[33345]: pgmap v1249: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:17.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:16 smithi131 ceph-mon[41654]: pgmap v1249: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:16 smithi131 ceph-mon[29507]: pgmap v1249: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:19.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:18 smithi157 ceph-mon[33345]: pgmap v1250: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:19.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:18 smithi131 ceph-mon[41654]: pgmap v1250: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:19.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:18 smithi131 ceph-mon[29507]: pgmap v1250: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:19.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:09:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:09:21.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:20 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:21.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:20 smithi157 ceph-mon[33345]: pgmap v1251: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:21.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:20 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:21.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:20 smithi131 ceph-mon[41654]: pgmap v1251: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:20 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:20 smithi131 ceph-mon[29507]: pgmap v1251: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:23.069 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:22 smithi131 ceph-mon[41654]: pgmap v1252: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:23.070 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:22 smithi131 ceph-mon[29507]: pgmap v1252: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:23.070 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:09:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:09:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:09:23.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:22 smithi157 ceph-mon[33345]: pgmap v1252: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:25.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:24 smithi157 ceph-mon[33345]: pgmap v1253: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:25.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:24 smithi131 ceph-mon[41654]: pgmap v1253: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:25.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:24 smithi131 ceph-mon[29507]: pgmap v1253: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:27.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:26 smithi157 ceph-mon[33345]: pgmap v1254: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:27.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:26 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:09:27.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:26 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:09:27.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:26 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:09:27.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:26 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:09:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:26 smithi131 ceph-mon[41654]: pgmap v1254: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:26 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:09:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:26 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:09:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:26 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:09:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:26 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:09:27.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:26 smithi131 ceph-mon[29507]: pgmap v1254: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:27.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:26 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:09:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:26 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:09:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:26 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:09:27.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:26 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:09:29.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:28 smithi157 ceph-mon[33345]: pgmap v1255: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:29.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:28 smithi131 ceph-mon[41654]: pgmap v1255: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:29.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:28 smithi131 ceph-mon[29507]: pgmap v1255: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:29.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:09:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:09:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:30 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:30 smithi157 ceph-mon[33345]: pgmap v1256: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:30 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:30 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:30 smithi131 ceph-mon[41654]: pgmap v1256: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:30 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:30 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:30 smithi131 ceph-mon[29507]: pgmap v1256: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:30 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:33.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:32 smithi157 ceph-mon[33345]: pgmap v1257: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:32 smithi131 ceph-mon[41654]: pgmap v1257: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:33.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:32 smithi131 ceph-mon[29507]: pgmap v1257: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:09:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:09:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:09:35.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:34 smithi157 ceph-mon[33345]: pgmap v1258: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:34 smithi131 ceph-mon[41654]: pgmap v1258: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:35.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:34 smithi131 ceph-mon[29507]: pgmap v1258: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:37.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:36 smithi157 ceph-mon[33345]: pgmap v1259: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:37.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:36 smithi131 ceph-mon[41654]: pgmap v1259: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:37.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:36 smithi131 ceph-mon[29507]: pgmap v1259: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:39.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:38 smithi157 ceph-mon[33345]: pgmap v1260: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:39.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:38 smithi131 ceph-mon[41654]: pgmap v1260: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:39.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:38 smithi131 ceph-mon[29507]: pgmap v1260: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:39.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:09:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:09:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:40 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:40 smithi157 ceph-mon[33345]: pgmap v1261: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:40 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:40 smithi131 ceph-mon[41654]: pgmap v1261: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:40 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:40 smithi131 ceph-mon[29507]: pgmap v1261: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:43.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:42 smithi157 ceph-mon[33345]: pgmap v1262: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:42 smithi131 ceph-mon[41654]: pgmap v1262: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:43.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:42 smithi131 ceph-mon[29507]: pgmap v1262: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:09:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:09:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:09:45.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:44 smithi157 ceph-mon[33345]: pgmap v1263: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:45.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:44 smithi131 ceph-mon[41654]: pgmap v1263: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:45.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:44 smithi131 ceph-mon[29507]: pgmap v1263: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:46.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:45 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:45 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:46.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:45 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:47.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:46 smithi157 ceph-mon[33345]: pgmap v1264: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:47.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:46 smithi131 ceph-mon[41654]: pgmap v1264: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:47.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:46 smithi131 ceph-mon[29507]: pgmap v1264: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:49.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:48 smithi157 ceph-mon[33345]: pgmap v1265: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:49.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:48 smithi131 ceph-mon[41654]: pgmap v1265: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:49.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:48 smithi131 ceph-mon[29507]: pgmap v1265: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:49.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:09:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:09:51.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:50 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:51.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:50 smithi157 ceph-mon[33345]: pgmap v1266: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:50 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:50 smithi131 ceph-mon[41654]: pgmap v1266: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:50 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:50 smithi131 ceph-mon[29507]: pgmap v1266: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:53.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:52 smithi157 ceph-mon[33345]: pgmap v1267: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:52 smithi131 ceph-mon[41654]: pgmap v1267: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:09:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:09:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:09:53.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:52 smithi131 ceph-mon[29507]: pgmap v1267: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:55.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:54 smithi157 ceph-mon[33345]: pgmap v1268: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:54 smithi131 ceph-mon[41654]: pgmap v1268: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:55.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:54 smithi131 ceph-mon[29507]: pgmap v1268: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:57.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:56 smithi157 ceph-mon[33345]: pgmap v1269: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:56 smithi131 ceph-mon[41654]: pgmap v1269: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:56 smithi131 ceph-mon[29507]: pgmap v1269: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:09:59.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:09:58 smithi157 ceph-mon[33345]: pgmap v1270: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:59.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:09:58 smithi131 ceph-mon[41654]: pgmap v1270: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:59.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:09:58 smithi131 ceph-mon[29507]: pgmap v1270: 228 pgs: 228 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:59.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:09:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:10:01.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:00 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:01.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:00 smithi157 ceph-mon[33345]: pgmap v1271: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:01.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:00 smithi157 ceph-mon[33345]: overall HEALTH_WARN 3 pool(s) do not have an application enabled 2024-02-05T08:10:01.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:10:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:00 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:00 smithi131 ceph-mon[41654]: pgmap v1271: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:00 smithi131 ceph-mon[41654]: overall HEALTH_WARN 3 pool(s) do not have an application enabled 2024-02-05T08:10:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:10:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:00 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:00 smithi131 ceph-mon[29507]: pgmap v1271: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:00 smithi131 ceph-mon[29507]: overall HEALTH_WARN 3 pool(s) do not have an application enabled 2024-02-05T08:10:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:10:03.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:02 smithi157 ceph-mon[33345]: pgmap v1272: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:02 smithi131 ceph-mon[41654]: pgmap v1272: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:02 smithi131 ceph-mon[29507]: pgmap v1272: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:10:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:10:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:10:05.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:04 smithi157 ceph-mon[33345]: pgmap v1273: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:04 smithi131 ceph-mon[41654]: pgmap v1273: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:05.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:04 smithi131 ceph-mon[29507]: pgmap v1273: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:07.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:06 smithi157 ceph-mon[33345]: pgmap v1274: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:07.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:06 smithi131 ceph-mon[41654]: pgmap v1274: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:07.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:06 smithi131 ceph-mon[29507]: pgmap v1274: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:09.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:08 smithi157 ceph-mon[33345]: pgmap v1275: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:09.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:08 smithi131 ceph-mon[41654]: pgmap v1275: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:09.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:08 smithi131 ceph-mon[29507]: pgmap v1275: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:09.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:10:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:10:11.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:10 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:11.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:10 smithi157 ceph-mon[33345]: pgmap v1276: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:10 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:10 smithi131 ceph-mon[41654]: pgmap v1276: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:10 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:10 smithi131 ceph-mon[29507]: pgmap v1276: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:10:13.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:12 smithi157 ceph-mon[33345]: pgmap v1277: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:12 smithi131 ceph-mon[41654]: pgmap v1277: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:12 smithi131 ceph-mon[29507]: pgmap v1277: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:10:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:10:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:10:15.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:14 smithi157 ceph-mon[33345]: pgmap v1278: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:15.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:14 smithi131 ceph-mon[41654]: pgmap v1278: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:15.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:14 smithi131 ceph-mon[29507]: pgmap v1278: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:16.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:10:16.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:10:16.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:10:17.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:16 smithi157 ceph-mon[33345]: pgmap v1279: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:17.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:16 smithi131 ceph-mon[41654]: pgmap v1279: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:16 smithi131 ceph-mon[29507]: pgmap v1279: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:17.787 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.787 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.787 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.787 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.787 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.787 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.787 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.787 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.787 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.787 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.787 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.788 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.788 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.788 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.788 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.788 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.788 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.788 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.788 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.789 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.789 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.789 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.789 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.789 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.789 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.789 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.789 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.789 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.789 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.790 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.790 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.790 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.790 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.790 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.790 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.790 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.790 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.790 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.790 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.791 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.791 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.791 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.791 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.791 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.791 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.791 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.791 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.791 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.791 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.791 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.792 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.792 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.792 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.792 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.792 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.792 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.792 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.792 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.792 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.793 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.793 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.793 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.793 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.793 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.793 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.793 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.793 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.793 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.793 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.793 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.794 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.794 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.794 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.794 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.794 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.794 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.794 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.794 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.794 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.794 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.794 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.794 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.794 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.795 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.795 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.795 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.795 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.795 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.795 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.795 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.795 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.795 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.795 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.795 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.795 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.796 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.796 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.796 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.796 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:17.796 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:10:19.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:18 smithi157 ceph-mon[33345]: pgmap v1280: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:19.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:18 smithi131 ceph-mon[41654]: pgmap v1280: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:19.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:18 smithi131 ceph-mon[29507]: pgmap v1280: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:19.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:10:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:10:21.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:20 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:21.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:20 smithi157 ceph-mon[33345]: pgmap v1281: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:21.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:20 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:21.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:20 smithi131 ceph-mon[41654]: pgmap v1281: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:20 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:20 smithi131 ceph-mon[29507]: pgmap v1281: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:23.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:22 smithi157 ceph-mon[33345]: pgmap v1282: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:22 smithi131 ceph-mon[41654]: pgmap v1282: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:23.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:22 smithi131 ceph-mon[29507]: pgmap v1282: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:10:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:10:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:10:25.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:24 smithi157 ceph-mon[33345]: pgmap v1283: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:25.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:24 smithi131 ceph-mon[41654]: pgmap v1283: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:25.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:24 smithi131 ceph-mon[29507]: pgmap v1283: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:27.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:26 smithi157 ceph-mon[33345]: pgmap v1284: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:27.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:26 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:10:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:26 smithi131 ceph-mon[41654]: pgmap v1284: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:26 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:10:27.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:26 smithi131 ceph-mon[29507]: pgmap v1284: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:27.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:26 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:10:28.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:10:28.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:27 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:10:28.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:27 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:10:28.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:10:28.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:27 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:10:28.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:27 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:10:28.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:10:28.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:27 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:10:28.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:27 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:10:29.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:28 smithi157 ceph-mon[33345]: pgmap v1285: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:29.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:28 smithi131 ceph-mon[41654]: pgmap v1285: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:29.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:28 smithi131 ceph-mon[29507]: pgmap v1285: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:29.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:10:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:10:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:30 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:30 smithi157 ceph-mon[33345]: pgmap v1286: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:30 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:10:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:30 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:30 smithi131 ceph-mon[41654]: pgmap v1286: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:30 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:10:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:30 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:30 smithi131 ceph-mon[29507]: pgmap v1286: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:30 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:10:33.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:32 smithi157 ceph-mon[33345]: pgmap v1287: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:32 smithi131 ceph-mon[41654]: pgmap v1287: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:33.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:32 smithi131 ceph-mon[29507]: pgmap v1287: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:33.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:10:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:10:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:10:35.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:34 smithi157 ceph-mon[33345]: pgmap v1288: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:34 smithi131 ceph-mon[41654]: pgmap v1288: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:35.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:34 smithi131 ceph-mon[29507]: pgmap v1288: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:37.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:36 smithi157 ceph-mon[33345]: pgmap v1289: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:37.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:36 smithi131 ceph-mon[41654]: pgmap v1289: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:37.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:36 smithi131 ceph-mon[29507]: pgmap v1289: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:39.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:38 smithi157 ceph-mon[33345]: pgmap v1290: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:39.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:38 smithi131 ceph-mon[41654]: pgmap v1290: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:39.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:38 smithi131 ceph-mon[29507]: pgmap v1290: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:39.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:10:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:10:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:40 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:40 smithi157 ceph-mon[33345]: pgmap v1291: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:40 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:40 smithi131 ceph-mon[41654]: pgmap v1291: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:40 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:40 smithi131 ceph-mon[29507]: pgmap v1291: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:43.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:42 smithi157 ceph-mon[33345]: pgmap v1292: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:42 smithi131 ceph-mon[41654]: pgmap v1292: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:43.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:42 smithi131 ceph-mon[29507]: pgmap v1292: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:10:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:10:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:10:45.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:44 smithi157 ceph-mon[33345]: pgmap v1293: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:45.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:44 smithi131 ceph-mon[41654]: pgmap v1293: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:45.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:44 smithi131 ceph-mon[29507]: pgmap v1293: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:46.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:45 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:10:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:45 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:10:46.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:45 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:10:47.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:46 smithi157 ceph-mon[33345]: pgmap v1294: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:47.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:46 smithi131 ceph-mon[41654]: pgmap v1294: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:47.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:46 smithi131 ceph-mon[29507]: pgmap v1294: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:49.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:48 smithi131 ceph-mon[41654]: pgmap v1295: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:49.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:48 smithi131 ceph-mon[29507]: pgmap v1295: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:49.343 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:48 smithi157 ceph-mon[33345]: pgmap v1295: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:49.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:10:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:10:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:50 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:50 smithi131 ceph-mon[41654]: pgmap v1296: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:50 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:50 smithi131 ceph-mon[29507]: pgmap v1296: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:50 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:50 smithi157 ceph-mon[33345]: pgmap v1296: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:52 smithi131 ceph-mon[41654]: pgmap v1297: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:53.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:52 smithi131 ceph-mon[29507]: pgmap v1297: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:10:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:10:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:10:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:52 smithi157 ceph-mon[33345]: pgmap v1297: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:54 smithi131 ceph-mon[41654]: pgmap v1298: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:55.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:54 smithi131 ceph-mon[29507]: pgmap v1298: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:54 smithi157 ceph-mon[33345]: pgmap v1298: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:56 smithi131 ceph-mon[41654]: pgmap v1299: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:56 smithi131 ceph-mon[29507]: pgmap v1299: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:56 smithi157 ceph-mon[33345]: pgmap v1299: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:10:59.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:10:58 smithi131 ceph-mon[41654]: pgmap v1300: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:59.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:10:58 smithi131 ceph-mon[29507]: pgmap v1300: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:59.272 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:10:58 smithi157 ceph-mon[33345]: pgmap v1300: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:59.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:10:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:11:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:00 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:00 smithi131 ceph-mon[41654]: pgmap v1301: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:11:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:00 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:00 smithi131 ceph-mon[29507]: pgmap v1301: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:11:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:00 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:00 smithi157 ceph-mon[33345]: pgmap v1301: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:11:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:02 smithi131 ceph-mon[41654]: pgmap v1302: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:02 smithi131 ceph-mon[29507]: pgmap v1302: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:11:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:11:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:11:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:02 smithi157 ceph-mon[33345]: pgmap v1302: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:05.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:05 smithi131 ceph-mon[29507]: pgmap v1303: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:05 smithi131 ceph-mon[41654]: pgmap v1303: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:05 smithi157 ceph-mon[33345]: pgmap v1303: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:07.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:07 smithi131 ceph-mon[29507]: pgmap v1304: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:07.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:07 smithi131 ceph-mon[41654]: pgmap v1304: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:07 smithi157 ceph-mon[33345]: pgmap v1304: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:09.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:09 smithi131 ceph-mon[41654]: pgmap v1305: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:09.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:09 smithi131 ceph-mon[29507]: pgmap v1305: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:09.354 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:09 smithi157 ceph-mon[33345]: pgmap v1305: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:09.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:11:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:11:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:11 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:11 smithi131 ceph-mon[41654]: pgmap v1306: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:11 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:11 smithi131 ceph-mon[29507]: pgmap v1306: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:11 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:11 smithi157 ceph-mon[33345]: pgmap v1306: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:13 smithi131 ceph-mon[41654]: pgmap v1307: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:13 smithi131 ceph-mon[29507]: pgmap v1307: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:11:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:11:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:11:13.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:13 smithi157 ceph-mon[33345]: pgmap v1307: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:15 smithi157 ceph-mon[33345]: pgmap v1308: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:15.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:15 smithi131 ceph-mon[41654]: pgmap v1308: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:15.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:15 smithi131 ceph-mon[29507]: pgmap v1308: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:16.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:11:16.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:11:16.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:11:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:17 smithi157 ceph-mon[33345]: pgmap v1309: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:17.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:17 smithi131 ceph-mon[41654]: pgmap v1309: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:17 smithi131 ceph-mon[29507]: pgmap v1309: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:19.365 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:19 smithi157 ceph-mon[33345]: pgmap v1310: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:19.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:19 smithi131 ceph-mon[41654]: pgmap v1310: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:19.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:19 smithi131 ceph-mon[29507]: pgmap v1310: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:19.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:11:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:11:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:21 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:21 smithi157 ceph-mon[33345]: pgmap v1311: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:21 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:21 smithi131 ceph-mon[41654]: pgmap v1311: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:21 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:21 smithi131 ceph-mon[29507]: pgmap v1311: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:23 smithi131 ceph-mon[41654]: pgmap v1312: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:23.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:23 smithi131 ceph-mon[29507]: pgmap v1312: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:11:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:11:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:11:23.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:23 smithi157 ceph-mon[33345]: pgmap v1312: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:25.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:25 smithi157 ceph-mon[33345]: pgmap v1313: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:25 smithi131 ceph-mon[41654]: pgmap v1313: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:25.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:25 smithi131 ceph-mon[29507]: pgmap v1313: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:27 smithi157 ceph-mon[33345]: pgmap v1314: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:27.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:27 smithi131 ceph-mon[41654]: pgmap v1314: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:27.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:27 smithi131 ceph-mon[29507]: pgmap v1314: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:28.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:28 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:11:28.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:28 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:11:28.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:28 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:11:29.371 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:29 smithi157 ceph-mon[33345]: pgmap v1315: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:29.372 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:29 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:11:29.372 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:29 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:11:29.372 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:29 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:11:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:29 smithi131 ceph-mon[41654]: pgmap v1315: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:29 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:11:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:29 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:11:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:29 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:11:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:29 smithi131 ceph-mon[29507]: pgmap v1315: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:29 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:11:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:29 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:11:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:29 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:11:29.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:11:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:11:29.720 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 08:11:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=cleanup t=2024-02-05T08:11:29.495636748Z level=info msg="Completed cleanup jobs" duration=4.442746ms 2024-02-05T08:11:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:31 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:31 smithi157 ceph-mon[33345]: pgmap v1316: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:31 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:11:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:31 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:31 smithi131 ceph-mon[41654]: pgmap v1316: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:31 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:11:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:31 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:31 smithi131 ceph-mon[29507]: pgmap v1316: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:31 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:11:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:33 smithi131 ceph-mon[41654]: pgmap v1317: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:33.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:33 smithi131 ceph-mon[29507]: pgmap v1317: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:11:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:11:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:11:33.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:33 smithi157 ceph-mon[33345]: pgmap v1317: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:35 smithi157 ceph-mon[33345]: pgmap v1318: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:35.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:35 smithi131 ceph-mon[41654]: pgmap v1318: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:35.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:35 smithi131 ceph-mon[29507]: pgmap v1318: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:37 smithi157 ceph-mon[33345]: pgmap v1319: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:37.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:37 smithi131 ceph-mon[41654]: pgmap v1319: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:37.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:37 smithi131 ceph-mon[29507]: pgmap v1319: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:39.386 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:39 smithi157 ceph-mon[33345]: pgmap v1320: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:39.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:39 smithi131 ceph-mon[41654]: pgmap v1320: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:39.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:39 smithi131 ceph-mon[29507]: pgmap v1320: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:39.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:11:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:11:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:41 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:41 smithi157 ceph-mon[33345]: pgmap v1321: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:41 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:41.551 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:41 smithi131 ceph-mon[29507]: pgmap v1321: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:41.552 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:41 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:41.552 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:41 smithi131 ceph-mon[41654]: pgmap v1321: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:43 smithi131 ceph-mon[41654]: pgmap v1322: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:43.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:43 smithi131 ceph-mon[29507]: pgmap v1322: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:11:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:11:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:11:43.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:43 smithi157 ceph-mon[33345]: pgmap v1322: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:45.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:45 smithi157 ceph-mon[33345]: pgmap v1323: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:45 smithi131 ceph-mon[41654]: pgmap v1323: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:45.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:45 smithi131 ceph-mon[29507]: pgmap v1323: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:46.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:46 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:11:46.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:46 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:11:46.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:46 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:11:47.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:47 smithi157 ceph-mon[33345]: pgmap v1324: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:47.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:47 smithi131 ceph-mon[41654]: pgmap v1324: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:47.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:47 smithi131 ceph-mon[29507]: pgmap v1324: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:49.397 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:49 smithi157 ceph-mon[33345]: pgmap v1325: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:49.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:49 smithi131 ceph-mon[41654]: pgmap v1325: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:49.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:49 smithi131 ceph-mon[29507]: pgmap v1325: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:49.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:11:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:11:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:51 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:51 smithi157 ceph-mon[33345]: pgmap v1326: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:51.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:51 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:51.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:51 smithi131 ceph-mon[41654]: pgmap v1326: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:51 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:11:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:51 smithi131 ceph-mon[29507]: pgmap v1326: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:53 smithi131 ceph-mon[41654]: pgmap v1327: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:53.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:53 smithi131 ceph-mon[29507]: pgmap v1327: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:53.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:11:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:11:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:11:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:53 smithi157 ceph-mon[33345]: pgmap v1327: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:55 smithi157 ceph-mon[33345]: pgmap v1328: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:55 smithi131 ceph-mon[41654]: pgmap v1328: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:55.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:55 smithi131 ceph-mon[29507]: pgmap v1328: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:11:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:57 smithi157 ceph-mon[33345]: pgmap v1329: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:57.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:57 smithi131 ceph-mon[41654]: pgmap v1329: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:57 smithi131 ceph-mon[29507]: pgmap v1329: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:11:57.798 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.798 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.798 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.798 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.798 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.799 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.799 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.799 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.799 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.799 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.799 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.799 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.799 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.799 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.799 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.799 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.799 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.799 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.799 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.799 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.800 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.800 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.800 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.800 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.800 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.800 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.800 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.800 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.800 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.800 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.800 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.800 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.800 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.800 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.800 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.801 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.801 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.801 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.801 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.801 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.801 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.801 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.801 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.801 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.801 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.801 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.801 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.801 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.801 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.801 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.802 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.803 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.803 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.803 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.803 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.803 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.803 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.803 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.803 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.803 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.803 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.803 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.803 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.803 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.803 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.803 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.803 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.804 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.804 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.804 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.804 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.804 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.804 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.804 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.804 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.804 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.804 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.804 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.804 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.804 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.804 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.804 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.805 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:57.805 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:11:59.408 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:11:59 smithi157 ceph-mon[33345]: pgmap v1330: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:59.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:11:59 smithi131 ceph-mon[41654]: pgmap v1330: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:59.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:11:59 smithi131 ceph-mon[29507]: pgmap v1330: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:11:59.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:11:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:12:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:01 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:01 smithi157 ceph-mon[33345]: pgmap v1331: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:01 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:12:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:01 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:01 smithi131 ceph-mon[41654]: pgmap v1331: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:01 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:12:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:01 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:01 smithi131 ceph-mon[29507]: pgmap v1331: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:01 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:12:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:03 smithi131 ceph-mon[41654]: pgmap v1332: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:03 smithi131 ceph-mon[29507]: pgmap v1332: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:12:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:12:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:12:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:03 smithi157 ceph-mon[33345]: pgmap v1332: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:05 smithi157 ceph-mon[33345]: pgmap v1333: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:05.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:05 smithi131 ceph-mon[41654]: pgmap v1333: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:05.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:05 smithi131 ceph-mon[29507]: pgmap v1333: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:07 smithi157 ceph-mon[33345]: pgmap v1334: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:07.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:07 smithi131 ceph-mon[41654]: pgmap v1334: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:07.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:07 smithi131 ceph-mon[29507]: pgmap v1334: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:09.415 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:09 smithi157 ceph-mon[33345]: pgmap v1335: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:09.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:09 smithi131 ceph-mon[41654]: pgmap v1335: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:09.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:09 smithi131 ceph-mon[29507]: pgmap v1335: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:09.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:12:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:12:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:11 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:11 smithi157 ceph-mon[33345]: pgmap v1336: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:11 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:11 smithi131 ceph-mon[41654]: pgmap v1336: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:11 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:11 smithi131 ceph-mon[29507]: pgmap v1336: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:13 smithi131 ceph-mon[41654]: pgmap v1337: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:12:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:12:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:12:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:13 smithi131 ceph-mon[29507]: pgmap v1337: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:13.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:13 smithi157 ceph-mon[33345]: pgmap v1337: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:15 smithi157 ceph-mon[33345]: pgmap v1338: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:15.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:15 smithi131 ceph-mon[41654]: pgmap v1338: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:15.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:15 smithi131 ceph-mon[29507]: pgmap v1338: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:16.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:12:16.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:12:16.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:12:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:17 smithi157 ceph-mon[33345]: pgmap v1339: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:17.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:17 smithi131 ceph-mon[41654]: pgmap v1339: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:17 smithi131 ceph-mon[29507]: pgmap v1339: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:19.425 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:19 smithi157 ceph-mon[33345]: pgmap v1340: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:19.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:19 smithi131 ceph-mon[41654]: pgmap v1340: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:19.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:19 smithi131 ceph-mon[29507]: pgmap v1340: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:19.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:12:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:12:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:21 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:21 smithi157 ceph-mon[33345]: pgmap v1341: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:21 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:21 smithi131 ceph-mon[29507]: pgmap v1341: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:21 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:21 smithi131 ceph-mon[41654]: pgmap v1341: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:12:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:12:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:12:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:23 smithi131 ceph-mon[41654]: pgmap v1342: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:23.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:23 smithi131 ceph-mon[29507]: pgmap v1342: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:23.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:23 smithi157 ceph-mon[33345]: pgmap v1342: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:25.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:25 smithi157 ceph-mon[33345]: pgmap v1343: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:25.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:25 smithi131 ceph-mon[41654]: pgmap v1343: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:25.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:25 smithi131 ceph-mon[29507]: pgmap v1343: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:27 smithi157 ceph-mon[33345]: pgmap v1344: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:27.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:27 smithi131 ceph-mon[41654]: pgmap v1344: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:27.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:27 smithi131 ceph-mon[29507]: pgmap v1344: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:29.436 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:29 smithi157 ceph-mon[33345]: pgmap v1345: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:29.437 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:29 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:12:29.437 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:29 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:12:29.437 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:29 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:12:29.437 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:29 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:12:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:29 smithi131 ceph-mon[29507]: pgmap v1345: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:29 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:12:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:29 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:12:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:29 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:12:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:29 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:12:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:29 smithi131 ceph-mon[41654]: pgmap v1345: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:29 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:12:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:29 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:12:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:29 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:12:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:29 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:12:29.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:12:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:12:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:31 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:31 smithi157 ceph-mon[33345]: pgmap v1346: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:31 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:12:31.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:31 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:31 smithi131 ceph-mon[41654]: pgmap v1346: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:31 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:12:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:31 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:31 smithi131 ceph-mon[29507]: pgmap v1346: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:31 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:12:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:33 smithi131 ceph-mon[41654]: pgmap v1347: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:33.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:33 smithi131 ceph-mon[29507]: pgmap v1347: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:12:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:12:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:12:33.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:33 smithi157 ceph-mon[33345]: pgmap v1347: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:35 smithi157 ceph-mon[33345]: pgmap v1348: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:35.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:35 smithi131 ceph-mon[29507]: pgmap v1348: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:35.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:35 smithi131 ceph-mon[41654]: pgmap v1348: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:37 smithi157 ceph-mon[33345]: pgmap v1349: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:37.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:37 smithi131 ceph-mon[41654]: pgmap v1349: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:37.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:37 smithi131 ceph-mon[29507]: pgmap v1349: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:38.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:38 smithi157 ceph-mon[33345]: pgmap v1350: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:38.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:38 smithi131 ceph-mon[41654]: pgmap v1350: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:38.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:38 smithi131 ceph-mon[29507]: pgmap v1350: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:39.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:12:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:12:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:40 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:40 smithi157 ceph-mon[33345]: pgmap v1351: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:41.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:40 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:40 smithi131 ceph-mon[41654]: pgmap v1351: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:40 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:40 smithi131 ceph-mon[29507]: pgmap v1351: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:43.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:42 smithi157 ceph-mon[33345]: pgmap v1352: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:42 smithi131 ceph-mon[41654]: pgmap v1352: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:43.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:42 smithi131 ceph-mon[29507]: pgmap v1352: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:12:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:12:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:12:45.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:44 smithi157 ceph-mon[33345]: pgmap v1353: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:45.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:44 smithi131 ceph-mon[41654]: pgmap v1353: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:45.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:44 smithi131 ceph-mon[29507]: pgmap v1353: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:46.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:45 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:12:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:45 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:12:46.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:45 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:12:47.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:46 smithi157 ceph-mon[33345]: pgmap v1354: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:47.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:46 smithi131 ceph-mon[41654]: pgmap v1354: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:47.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:46 smithi131 ceph-mon[29507]: pgmap v1354: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:49.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:48 smithi157 ceph-mon[33345]: pgmap v1355: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:49.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:48 smithi131 ceph-mon[41654]: pgmap v1355: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:49.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:48 smithi131 ceph-mon[29507]: pgmap v1355: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:49.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:12:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:12:51.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:50 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:51.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:50 smithi157 ceph-mon[33345]: pgmap v1356: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:50 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:50 smithi131 ceph-mon[41654]: pgmap v1356: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:50 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:12:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:50 smithi131 ceph-mon[29507]: pgmap v1356: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:53.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:52 smithi157 ceph-mon[33345]: pgmap v1357: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:52 smithi131 ceph-mon[41654]: pgmap v1357: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:12:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:12:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:12:53.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:52 smithi131 ceph-mon[29507]: pgmap v1357: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:55.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:54 smithi157 ceph-mon[33345]: pgmap v1358: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:54 smithi131 ceph-mon[41654]: pgmap v1358: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:55.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:54 smithi131 ceph-mon[29507]: pgmap v1358: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:12:57.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:56 smithi157 ceph-mon[33345]: pgmap v1359: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:57.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:56 smithi131 ceph-mon[41654]: pgmap v1359: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:56 smithi131 ceph-mon[29507]: pgmap v1359: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:12:59.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:12:58 smithi157 ceph-mon[33345]: pgmap v1360: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:59.266 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:12:58 smithi131 ceph-mon[29507]: pgmap v1360: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:59.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:12:58 smithi131 ceph-mon[41654]: pgmap v1360: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:12:59.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:12:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:13:01.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:00 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:01.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:00 smithi157 ceph-mon[33345]: pgmap v1361: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:01.221 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:13:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:00 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:00 smithi131 ceph-mon[41654]: pgmap v1361: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:13:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:00 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:00 smithi131 ceph-mon[29507]: pgmap v1361: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:13:03.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:02 smithi157 ceph-mon[33345]: pgmap v1362: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:13:03.266 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:02 smithi131 ceph-mon[29507]: pgmap v1362: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:13:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:13:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:13:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:13:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:02 smithi131 ceph-mon[41654]: pgmap v1362: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:13:05.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:04 smithi157 ceph-mon[33345]: pgmap v1363: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:05.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:04 smithi131 ceph-mon[29507]: pgmap v1363: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:04 smithi131 ceph-mon[41654]: pgmap v1363: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:07.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:06 smithi157 ceph-mon[33345]: pgmap v1364: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:13:07.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:06 smithi131 ceph-mon[41654]: pgmap v1364: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:13:07.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:06 smithi131 ceph-mon[29507]: pgmap v1364: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:13:09.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:08 smithi157 ceph-mon[33345]: pgmap v1365: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:13:09.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:08 smithi131 ceph-mon[41654]: pgmap v1365: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:13:09.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:08 smithi131 ceph-mon[29507]: pgmap v1365: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:13:09.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:13:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:13:11.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:10 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:11.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:10 smithi157 ceph-mon[33345]: pgmap v1366: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:10 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:10 smithi131 ceph-mon[41654]: pgmap v1366: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:10 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:10 smithi131 ceph-mon[29507]: pgmap v1366: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:13.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:12 smithi157 ceph-mon[33345]: pgmap v1367: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:12 smithi131 ceph-mon[41654]: pgmap v1367: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:12 smithi131 ceph-mon[29507]: pgmap v1367: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:13:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:13:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:13:15.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:14 smithi157 ceph-mon[33345]: pgmap v1368: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:15.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:14 smithi131 ceph-mon[41654]: pgmap v1368: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:15.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:14 smithi131 ceph-mon[29507]: pgmap v1368: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:16.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:13:16.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:13:16.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:13:17.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:16 smithi157 ceph-mon[33345]: pgmap v1369: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:17.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:16 smithi131 ceph-mon[41654]: pgmap v1369: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:16 smithi131 ceph-mon[29507]: pgmap v1369: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:19.223 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:18 smithi157 ceph-mon[33345]: pgmap v1370: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:19.272 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:18 smithi131 ceph-mon[41654]: pgmap v1370: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:19.273 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:18 smithi131 ceph-mon[29507]: pgmap v1370: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:19.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:13:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:13:21.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:20 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:21.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:20 smithi157 ceph-mon[33345]: pgmap v1371: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:21.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:20 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:21.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:20 smithi131 ceph-mon[41654]: pgmap v1371: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:20 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:20 smithi131 ceph-mon[29507]: pgmap v1371: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:23.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:22 smithi157 ceph-mon[33345]: pgmap v1372: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:22 smithi131 ceph-mon[41654]: pgmap v1372: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:13:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:13:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:13:23.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:22 smithi131 ceph-mon[29507]: pgmap v1372: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:25.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:24 smithi157 ceph-mon[33345]: pgmap v1373: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:25.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:24 smithi131 ceph-mon[41654]: pgmap v1373: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:25.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:24 smithi131 ceph-mon[29507]: pgmap v1373: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:27.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:26 smithi157 ceph-mon[33345]: pgmap v1374: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:26 smithi131 ceph-mon[41654]: pgmap v1374: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:27.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:26 smithi131 ceph-mon[29507]: pgmap v1374: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:29.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:28 smithi157 ceph-mon[33345]: pgmap v1375: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:29.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:28 smithi131 ceph-mon[41654]: pgmap v1375: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:29.266 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:28 smithi131 ceph-mon[29507]: pgmap v1375: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:29.866 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:13:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:13:30.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:29 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:13:30.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:29 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:13:30.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:29 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:13:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:30 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:30 smithi157 ceph-mon[33345]: pgmap v1376: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:30 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:13:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:30 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:30 smithi131 ceph-mon[29507]: pgmap v1376: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:30 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:13:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:30 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:30 smithi131 ceph-mon[41654]: pgmap v1376: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:30 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:13:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:13:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:13:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:13:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:33 smithi131 ceph-mon[41654]: pgmap v1377: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:33 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:13:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:33 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:13:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:33 smithi131 ceph-mon[29507]: pgmap v1377: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:33 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:13:33.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:33 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:13:33.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:33 smithi157 ceph-mon[33345]: pgmap v1377: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:33 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:13:33.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:33 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:13:34.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:34 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:13:34.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:34 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:13:34.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:34 smithi131 ceph-mon[41654]: pgmap v1378: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:34.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:34 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:13:34.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:34 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:13:34.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:34 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:13:34.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:34 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:13:34.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:34 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:13:34.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:34 smithi131 ceph-mon[29507]: pgmap v1378: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:34.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:34 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:13:34.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:34 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:13:34.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:34 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:13:34.970 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:34 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:13:34.970 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:34 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:13:34.970 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:34 smithi157 ceph-mon[33345]: pgmap v1378: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:34.970 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:34 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:13:34.970 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:34 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:13:34.970 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:34 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:13:37.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:36 smithi157 ceph-mon[33345]: pgmap v1379: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:37.266 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:36 smithi131 ceph-mon[29507]: pgmap v1379: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:37.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:36 smithi131 ceph-mon[41654]: pgmap v1379: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:37.811 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.811 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.811 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.811 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.811 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.811 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.811 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.811 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.811 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.811 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.811 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.812 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.812 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.812 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.812 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.812 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.812 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.812 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.812 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.812 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.812 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.812 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.813 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.813 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.813 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.813 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.813 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.813 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.813 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.813 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.813 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.813 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.813 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.813 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.813 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.813 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.814 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.814 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.814 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.814 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.814 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.814 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.814 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.814 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.814 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.814 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.814 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.814 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.814 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.814 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.815 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.815 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.815 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.815 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.815 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.815 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.815 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.815 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.815 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.815 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.815 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.815 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.815 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.815 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.816 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.816 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.816 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.816 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.816 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.816 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.816 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.816 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.816 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.816 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.816 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.817 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.817 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.817 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.817 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.817 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.817 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.817 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.817 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.817 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.817 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.817 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.817 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.817 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.817 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.818 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.818 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.818 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.818 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.818 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.818 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.818 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.818 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.818 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.818 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:37.818 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:13:39.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:38 smithi157 ceph-mon[33345]: pgmap v1380: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:39.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:38 smithi131 ceph-mon[41654]: pgmap v1380: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:39.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:38 smithi131 ceph-mon[29507]: pgmap v1380: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:39.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:13:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:13:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:40 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:40 smithi157 ceph-mon[33345]: pgmap v1381: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:41.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:40 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:40 smithi131 ceph-mon[41654]: pgmap v1381: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:40 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:40 smithi131 ceph-mon[29507]: pgmap v1381: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:43.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:42 smithi157 ceph-mon[33345]: pgmap v1382: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:42 smithi131 ceph-mon[41654]: pgmap v1382: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:43.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:42 smithi131 ceph-mon[29507]: pgmap v1382: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:13:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:13:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:13:45.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:44 smithi157 ceph-mon[33345]: pgmap v1383: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:45.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:44 smithi131 ceph-mon[41654]: pgmap v1383: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:45.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:44 smithi131 ceph-mon[29507]: pgmap v1383: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:46.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:45 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:13:46.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:45 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:13:46.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:45 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:13:47.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:46 smithi157 ceph-mon[33345]: pgmap v1384: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:47.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:46 smithi131 ceph-mon[41654]: pgmap v1384: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:47.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:46 smithi131 ceph-mon[29507]: pgmap v1384: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:49.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:48 smithi157 ceph-mon[33345]: pgmap v1385: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:49.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:48 smithi131 ceph-mon[41654]: pgmap v1385: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:49.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:48 smithi131 ceph-mon[29507]: pgmap v1385: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:49.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:13:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:13:51.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:50 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:51.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:50 smithi157 ceph-mon[33345]: pgmap v1386: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:51.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:50 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:50 smithi131 ceph-mon[41654]: pgmap v1386: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:50 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:13:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:50 smithi131 ceph-mon[29507]: pgmap v1386: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:53.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:52 smithi157 ceph-mon[33345]: pgmap v1387: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:53.266 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:52 smithi131 ceph-mon[29507]: pgmap v1387: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:52 smithi131 ceph-mon[41654]: pgmap v1387: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:13:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:13:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:13:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:13:55.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:54 smithi157 ceph-mon[33345]: pgmap v1388: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:55.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:54 smithi131 ceph-mon[41654]: pgmap v1388: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:55.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:54 smithi131 ceph-mon[29507]: pgmap v1388: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:13:57.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:56 smithi157 ceph-mon[33345]: pgmap v1389: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:56 smithi131 ceph-mon[41654]: pgmap v1389: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:56 smithi131 ceph-mon[29507]: pgmap v1389: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:59.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:13:58 smithi157 ceph-mon[33345]: pgmap v1390: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:59.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:13:58 smithi131 ceph-mon[41654]: pgmap v1390: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:59.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:13:58 smithi131 ceph-mon[29507]: pgmap v1390: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:13:59.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:13:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:14:01.216 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:00 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:01.216 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:00 smithi157 ceph-mon[33345]: pgmap v1391: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:01.216 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:14:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:00 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:00 smithi131 ceph-mon[41654]: pgmap v1391: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:14:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:00 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:00 smithi131 ceph-mon[29507]: pgmap v1391: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:14:03.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:02 smithi157 ceph-mon[33345]: pgmap v1392: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:03.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:02 smithi131 ceph-mon[41654]: pgmap v1392: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:02 smithi131 ceph-mon[29507]: pgmap v1392: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:14:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:14:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:14:05.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:04 smithi157 ceph-mon[33345]: pgmap v1393: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:05.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:04 smithi131 ceph-mon[41654]: pgmap v1393: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:05.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:04 smithi131 ceph-mon[29507]: pgmap v1393: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:07.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:06 smithi157 ceph-mon[33345]: pgmap v1394: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:07.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:06 smithi131 ceph-mon[41654]: pgmap v1394: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:07.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:06 smithi131 ceph-mon[29507]: pgmap v1394: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:09.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:08 smithi157 ceph-mon[33345]: pgmap v1395: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:09.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:08 smithi131 ceph-mon[41654]: pgmap v1395: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:09.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:08 smithi131 ceph-mon[29507]: pgmap v1395: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:09.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:14:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:14:11.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:10 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:11.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:10 smithi157 ceph-mon[33345]: pgmap v1396: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:11.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:10 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:10 smithi131 ceph-mon[41654]: pgmap v1396: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:10 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:10 smithi131 ceph-mon[29507]: pgmap v1396: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:13.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:12 smithi157 ceph-mon[33345]: pgmap v1397: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:13.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:12 smithi131 ceph-mon[41654]: pgmap v1397: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:12 smithi131 ceph-mon[29507]: pgmap v1397: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:14:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:14:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:14:15.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:14 smithi157 ceph-mon[33345]: pgmap v1398: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:15.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:14 smithi131 ceph-mon[41654]: pgmap v1398: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:15.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:14 smithi131 ceph-mon[29507]: pgmap v1398: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:16.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:14:16.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:14:16.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:14:17.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:16 smithi157 ceph-mon[33345]: pgmap v1399: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:17.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:16 smithi131 ceph-mon[41654]: pgmap v1399: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:16 smithi131 ceph-mon[29507]: pgmap v1399: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:19.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:18 smithi157 ceph-mon[33345]: pgmap v1400: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:19.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:18 smithi131 ceph-mon[41654]: pgmap v1400: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:19.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:18 smithi131 ceph-mon[29507]: pgmap v1400: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:19.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:14:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:14:21.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:20 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:21.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:20 smithi157 ceph-mon[33345]: pgmap v1401: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:21.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:20 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:21.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:20 smithi131 ceph-mon[41654]: pgmap v1401: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:20 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:20 smithi131 ceph-mon[29507]: pgmap v1401: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:23.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:22 smithi157 ceph-mon[33345]: pgmap v1402: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:23.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:22 smithi131 ceph-mon[41654]: pgmap v1402: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:23.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:22 smithi131 ceph-mon[29507]: pgmap v1402: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:14:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:14:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:14:25.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:24 smithi157 ceph-mon[33345]: pgmap v1403: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:25.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:24 smithi131 ceph-mon[41654]: pgmap v1403: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:25.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:24 smithi131 ceph-mon[29507]: pgmap v1403: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:27.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:26 smithi157 ceph-mon[33345]: pgmap v1404: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:26 smithi131 ceph-mon[41654]: pgmap v1404: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:27.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:26 smithi131 ceph-mon[29507]: pgmap v1404: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:29.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:28 smithi157 ceph-mon[33345]: pgmap v1405: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:29.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:28 smithi131 ceph-mon[29507]: pgmap v1405: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:29.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:28 smithi131 ceph-mon[41654]: pgmap v1405: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:29.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:14:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:14:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:30 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:30 smithi157 ceph-mon[33345]: pgmap v1406: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:31.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:30 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:14:31.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:30 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:30 smithi131 ceph-mon[41654]: pgmap v1406: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:30 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:14:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:30 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:30 smithi131 ceph-mon[29507]: pgmap v1406: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:30 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:14:33.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:32 smithi157 ceph-mon[33345]: pgmap v1407: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:32 smithi131 ceph-mon[41654]: pgmap v1407: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:33.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:32 smithi131 ceph-mon[29507]: pgmap v1407: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:14:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:14:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:14:35.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:34 smithi157 ceph-mon[33345]: pgmap v1408: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:35.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:34 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:14:35.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:34 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:14:35.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:34 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:14:35.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:34 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:14:35.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:34 smithi131 ceph-mon[29507]: pgmap v1408: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:35.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:34 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:14:35.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:34 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:14:35.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:34 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:14:35.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:34 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:14:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:34 smithi131 ceph-mon[41654]: pgmap v1408: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:35.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:34 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:14:35.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:34 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:14:35.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:34 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:14:35.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:34 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:14:37.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:36 smithi157 ceph-mon[33345]: pgmap v1409: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:37.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:36 smithi131 ceph-mon[41654]: pgmap v1409: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:37.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:36 smithi131 ceph-mon[29507]: pgmap v1409: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:39.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:38 smithi157 ceph-mon[33345]: pgmap v1410: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:39.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:38 smithi131 ceph-mon[41654]: pgmap v1410: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:39.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:38 smithi131 ceph-mon[29507]: pgmap v1410: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:39.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:14:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:14:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:40 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:40 smithi157 ceph-mon[33345]: pgmap v1411: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:40 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:40 smithi131 ceph-mon[41654]: pgmap v1411: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:40 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:40 smithi131 ceph-mon[29507]: pgmap v1411: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:43.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:42 smithi157 ceph-mon[33345]: pgmap v1412: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:43.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:42 smithi131 ceph-mon[41654]: pgmap v1412: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:43.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:42 smithi131 ceph-mon[29507]: pgmap v1412: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:14:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:14:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:14:45.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:44 smithi131 ceph-mon[41654]: pgmap v1413: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:45.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:44 smithi131 ceph-mon[29507]: pgmap v1413: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:45.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:44 smithi157 ceph-mon[33345]: pgmap v1413: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:45 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:14:46.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:45 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:14:46.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:45 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:14:47.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:46 smithi131 ceph-mon[41654]: pgmap v1414: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:47.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:46 smithi131 ceph-mon[29507]: pgmap v1414: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:47.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:46 smithi157 ceph-mon[33345]: pgmap v1414: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:49.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:48 smithi131 ceph-mon[41654]: pgmap v1415: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:49.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:48 smithi131 ceph-mon[29507]: pgmap v1415: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:49.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:48 smithi157 ceph-mon[33345]: pgmap v1415: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:14:49.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:14:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:14:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:50 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:50 smithi131 ceph-mon[41654]: pgmap v1416: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:50 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:50 smithi131 ceph-mon[29507]: pgmap v1416: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:50 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:14:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:50 smithi157 ceph-mon[33345]: pgmap v1416: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:52 smithi131 ceph-mon[41654]: pgmap v1417: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:53.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:52 smithi131 ceph-mon[29507]: pgmap v1417: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:14:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:14:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:14:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:52 smithi157 ceph-mon[33345]: pgmap v1417: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:54 smithi131 ceph-mon[41654]: pgmap v1418: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:55.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:54 smithi131 ceph-mon[29507]: pgmap v1418: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:54 smithi157 ceph-mon[33345]: pgmap v1418: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:14:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:56 smithi131 ceph-mon[41654]: pgmap v1419: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:56 smithi131 ceph-mon[29507]: pgmap v1419: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:56 smithi157 ceph-mon[33345]: pgmap v1419: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:59.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:14:58 smithi131 ceph-mon[41654]: pgmap v1420: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:59.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:14:58 smithi131 ceph-mon[29507]: pgmap v1420: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:59.272 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:14:58 smithi157 ceph-mon[33345]: pgmap v1420: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:14:59.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:14:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:15:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:01 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:01 smithi131 ceph-mon[41654]: pgmap v1421: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:01 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:15:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:01 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:01 smithi131 ceph-mon[29507]: pgmap v1421: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:01 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:15:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:01 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:01 smithi157 ceph-mon[33345]: pgmap v1421: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:01 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:15:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:03 smithi131 ceph-mon[41654]: pgmap v1422: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:03 smithi131 ceph-mon[29507]: pgmap v1422: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:15:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:15:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:15:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:03 smithi157 ceph-mon[33345]: pgmap v1422: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:05 smithi131 ceph-mon[41654]: pgmap v1423: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:05.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:05 smithi131 ceph-mon[29507]: pgmap v1423: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:05 smithi157 ceph-mon[33345]: pgmap v1423: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:07.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:07 smithi131 ceph-mon[29507]: pgmap v1424: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:07.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:07 smithi131 ceph-mon[41654]: pgmap v1424: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:07 smithi157 ceph-mon[33345]: pgmap v1424: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:09.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:09 smithi131 ceph-mon[29507]: pgmap v1425: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:09.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:09 smithi131 ceph-mon[41654]: pgmap v1425: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:09.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:09 smithi157 ceph-mon[33345]: pgmap v1425: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:09.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:15:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:15:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:11 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:11 smithi131 ceph-mon[29507]: pgmap v1426: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:11 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:11 smithi131 ceph-mon[41654]: pgmap v1426: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:11 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:11 smithi157 ceph-mon[33345]: pgmap v1426: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:13.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:13 smithi131 ceph-mon[41654]: pgmap v1427: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:13 smithi131 ceph-mon[29507]: pgmap v1427: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:15:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:15:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:15:13.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:13 smithi157 ceph-mon[33345]: pgmap v1427: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:15 smithi157 ceph-mon[33345]: pgmap v1428: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:15.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:15 smithi131 ceph-mon[41654]: pgmap v1428: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:15.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:15 smithi131 ceph-mon[29507]: pgmap v1428: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:16.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:15:16.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:15:16.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:15:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:17 smithi157 ceph-mon[33345]: pgmap v1429: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:17.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:17 smithi131 ceph-mon[41654]: pgmap v1429: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:17 smithi131 ceph-mon[29507]: pgmap v1429: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:17.822 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.822 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.823 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.823 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.823 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.823 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.823 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.823 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.823 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.823 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.823 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.823 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.823 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.823 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.823 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.823 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.823 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.824 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.824 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.824 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.824 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.824 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.824 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.824 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.824 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.824 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.824 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.824 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.824 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.824 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.824 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.824 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.825 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.825 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.825 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.825 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.825 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.825 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.825 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.825 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.825 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.825 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.825 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.825 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.825 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.825 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.825 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.825 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.826 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.826 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.826 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.826 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.826 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.826 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.826 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.826 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.826 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.826 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.826 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.826 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.826 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.826 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.826 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.826 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.827 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.828 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.828 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.828 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.828 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.828 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.828 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.828 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.828 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.828 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.828 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.828 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.828 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.828 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.828 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.829 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.829 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.829 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.829 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:17.829 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:15:19.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:19 smithi157 ceph-mon[33345]: pgmap v1430: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:19.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:19 smithi131 ceph-mon[41654]: pgmap v1430: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:19.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:19 smithi131 ceph-mon[29507]: pgmap v1430: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:19.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:15:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:15:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:21 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:21 smithi157 ceph-mon[33345]: pgmap v1431: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:21 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:21 smithi131 ceph-mon[41654]: pgmap v1431: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:21 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:21 smithi131 ceph-mon[29507]: pgmap v1431: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:23.266 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:23 smithi131 ceph-mon[29507]: pgmap v1432: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:23 smithi131 ceph-mon[41654]: pgmap v1432: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:15:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:15:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:15:23.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:23 smithi157 ceph-mon[33345]: pgmap v1432: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:25.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:25 smithi157 ceph-mon[33345]: pgmap v1433: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:25 smithi131 ceph-mon[41654]: pgmap v1433: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:25.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:25 smithi131 ceph-mon[29507]: pgmap v1433: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:27 smithi157 ceph-mon[33345]: pgmap v1434: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:27.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:27 smithi131 ceph-mon[41654]: pgmap v1434: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:27.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:27 smithi131 ceph-mon[29507]: pgmap v1434: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:29.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:29 smithi157 ceph-mon[33345]: pgmap v1435: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:29.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:29 smithi131 ceph-mon[41654]: pgmap v1435: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:29 smithi131 ceph-mon[29507]: pgmap v1435: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:29.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:15:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:15:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:31 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:31 smithi157 ceph-mon[33345]: pgmap v1436: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:31 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:15:31.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:31 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:31 smithi131 ceph-mon[41654]: pgmap v1436: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:31 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:15:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:31 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:31 smithi131 ceph-mon[29507]: pgmap v1436: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:31 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:15:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:33 smithi131 ceph-mon[41654]: pgmap v1437: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:15:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:15:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:15:33.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:33 smithi131 ceph-mon[29507]: pgmap v1437: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:33.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:33 smithi157 ceph-mon[33345]: pgmap v1437: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:35 smithi157 ceph-mon[33345]: pgmap v1438: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:35 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:15:35.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:35 smithi131 ceph-mon[41654]: pgmap v1438: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:35.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:35 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:15:35.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:35 smithi131 ceph-mon[29507]: pgmap v1438: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:35.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:35 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:15:36.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:36 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:15:36.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:36 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:15:36.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:36 smithi157 ceph-mon[33345]: pgmap v1439: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:36.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:36 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:15:36.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:36 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:15:36.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:36 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:15:36.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:36 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:15:36.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:36 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:15:36.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:36 smithi131 ceph-mon[41654]: pgmap v1439: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:36.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:36 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:15:36.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:36 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:15:36.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:36 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:15:36.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:36 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:15:36.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:36 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:15:36.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:36 smithi131 ceph-mon[29507]: pgmap v1439: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:36.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:36 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:15:36.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:36 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:15:36.768 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:36 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:15:39.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:38 smithi157 ceph-mon[33345]: pgmap v1440: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:39.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:38 smithi131 ceph-mon[41654]: pgmap v1440: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:39.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:38 smithi131 ceph-mon[29507]: pgmap v1440: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:39.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:15:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:15:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:40 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:40 smithi157 ceph-mon[33345]: pgmap v1441: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:40 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:40 smithi131 ceph-mon[41654]: pgmap v1441: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:40 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:40 smithi131 ceph-mon[29507]: pgmap v1441: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:43.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:42 smithi157 ceph-mon[33345]: pgmap v1442: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:43.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:42 smithi131 ceph-mon[41654]: pgmap v1442: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:15:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:15:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:15:43.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:42 smithi131 ceph-mon[29507]: pgmap v1442: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:45.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:44 smithi157 ceph-mon[33345]: pgmap v1443: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:45.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:44 smithi131 ceph-mon[41654]: pgmap v1443: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:45.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:44 smithi131 ceph-mon[29507]: pgmap v1443: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:46.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:45 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:15:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:45 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:15:46.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:45 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:15:47.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:46 smithi157 ceph-mon[33345]: pgmap v1444: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:47.266 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:46 smithi131 ceph-mon[29507]: pgmap v1444: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:47.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:46 smithi131 ceph-mon[41654]: pgmap v1444: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:49.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:48 smithi157 ceph-mon[33345]: pgmap v1445: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:49.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:48 smithi131 ceph-mon[41654]: pgmap v1445: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:49.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:48 smithi131 ceph-mon[29507]: pgmap v1445: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:49.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:15:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:15:51.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:50 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:51.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:50 smithi157 ceph-mon[33345]: pgmap v1446: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:50 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:50 smithi131 ceph-mon[29507]: pgmap v1446: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:50 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:15:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:50 smithi131 ceph-mon[41654]: pgmap v1446: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:53.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:52 smithi157 ceph-mon[33345]: pgmap v1447: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:52 smithi131 ceph-mon[41654]: pgmap v1447: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:53.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:52 smithi131 ceph-mon[29507]: pgmap v1447: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:15:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:15:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:15:55.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:54 smithi157 ceph-mon[33345]: pgmap v1448: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:55.266 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:54 smithi131 ceph-mon[29507]: pgmap v1448: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:54 smithi131 ceph-mon[41654]: pgmap v1448: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:15:57.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:56 smithi157 ceph-mon[33345]: pgmap v1449: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:56 smithi131 ceph-mon[41654]: pgmap v1449: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:56 smithi131 ceph-mon[29507]: pgmap v1449: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:15:59.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:15:58 smithi157 ceph-mon[33345]: pgmap v1450: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:59.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:15:58 smithi131 ceph-mon[41654]: pgmap v1450: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:59.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:15:58 smithi131 ceph-mon[29507]: pgmap v1450: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:15:59.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:15:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:16:01.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:00 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:01.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:00 smithi157 ceph-mon[33345]: pgmap v1451: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:01.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:16:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:00 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:00 smithi131 ceph-mon[41654]: pgmap v1451: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:16:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:00 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:00 smithi131 ceph-mon[29507]: pgmap v1451: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:16:03.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:02 smithi157 ceph-mon[33345]: pgmap v1452: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:03.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:02 smithi131 ceph-mon[41654]: pgmap v1452: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:02 smithi131 ceph-mon[29507]: pgmap v1452: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:16:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:16:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:16:05.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:04 smithi157 ceph-mon[33345]: pgmap v1453: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:05.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:04 smithi131 ceph-mon[29507]: pgmap v1453: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:04 smithi131 ceph-mon[41654]: pgmap v1453: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:07.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:06 smithi157 ceph-mon[33345]: pgmap v1454: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:07.266 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:06 smithi131 ceph-mon[29507]: pgmap v1454: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:07.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:06 smithi131 ceph-mon[41654]: pgmap v1454: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:09.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:08 smithi157 ceph-mon[33345]: pgmap v1455: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:09.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:08 smithi131 ceph-mon[41654]: pgmap v1455: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:09.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:08 smithi131 ceph-mon[29507]: pgmap v1455: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:09.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:16:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:16:11.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:10 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:11.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:10 smithi157 ceph-mon[33345]: pgmap v1456: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:11.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:10 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:10 smithi131 ceph-mon[41654]: pgmap v1456: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:10 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:10 smithi131 ceph-mon[29507]: pgmap v1456: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:13.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:12 smithi157 ceph-mon[33345]: pgmap v1457: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:13.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:12 smithi131 ceph-mon[41654]: pgmap v1457: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:12 smithi131 ceph-mon[29507]: pgmap v1457: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:16:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:16:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:16:15.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:14 smithi157 ceph-mon[33345]: pgmap v1458: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:15.266 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:14 smithi131 ceph-mon[29507]: pgmap v1458: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:15.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:14 smithi131 ceph-mon[41654]: pgmap v1458: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:16.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:15 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:16:16.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:15 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:16:16.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:15 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:16:17.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:16 smithi157 ceph-mon[33345]: pgmap v1459: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:17.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:16 smithi131 ceph-mon[41654]: pgmap v1459: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:16 smithi131 ceph-mon[29507]: pgmap v1459: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:19.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:18 smithi157 ceph-mon[33345]: pgmap v1460: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:19.266 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:18 smithi131 ceph-mon[29507]: pgmap v1460: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:19.279 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:18 smithi131 ceph-mon[41654]: pgmap v1460: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:19.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:16:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:16:21.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:20 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:21.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:20 smithi131 ceph-mon[41654]: pgmap v1461: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:20 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:20 smithi131 ceph-mon[29507]: pgmap v1461: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:20 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:20 smithi157 ceph-mon[33345]: pgmap v1461: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:22 smithi131 ceph-mon[41654]: pgmap v1462: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:16:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:16:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:16:23.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:22 smithi131 ceph-mon[29507]: pgmap v1462: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:23.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:22 smithi157 ceph-mon[33345]: pgmap v1462: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:25.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:24 smithi131 ceph-mon[41654]: pgmap v1463: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:25.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:24 smithi131 ceph-mon[29507]: pgmap v1463: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:25.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:24 smithi157 ceph-mon[33345]: pgmap v1463: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:27.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:26 smithi131 ceph-mon[41654]: pgmap v1464: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:27.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:26 smithi131 ceph-mon[29507]: pgmap v1464: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:26 smithi157 ceph-mon[33345]: pgmap v1464: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:29.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:28 smithi131 ceph-mon[41654]: pgmap v1465: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:29.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:28 smithi131 ceph-mon[29507]: pgmap v1465: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:29.272 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:28 smithi157 ceph-mon[33345]: pgmap v1465: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:29.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:16:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:16:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:30 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:30 smithi131 ceph-mon[41654]: pgmap v1466: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:31.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:30 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:16:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:30 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:30 smithi131 ceph-mon[29507]: pgmap v1466: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:31.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:30 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:16:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:30 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:30 smithi157 ceph-mon[33345]: pgmap v1466: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:30 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:16:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:32 smithi131 ceph-mon[41654]: pgmap v1467: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:33.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:32 smithi131 ceph-mon[29507]: pgmap v1467: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:16:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:16:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:16:33.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:32 smithi157 ceph-mon[33345]: pgmap v1467: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:34 smithi131 ceph-mon[41654]: pgmap v1468: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:35.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:34 smithi131 ceph-mon[29507]: pgmap v1468: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:34 smithi157 ceph-mon[33345]: pgmap v1468: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:37.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:36 smithi131 ceph-mon[41654]: pgmap v1469: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:37.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:36 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:16:37.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:36 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:16:37.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:36 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:16:37.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:36 smithi131 ceph-mon[29507]: pgmap v1469: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:37.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:36 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:16:37.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:36 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:16:37.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:36 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:16:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:36 smithi157 ceph-mon[33345]: pgmap v1469: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:36 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:16:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:36 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:16:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:36 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:16:38.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:37 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:16:38.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:37 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:16:38.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:37 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:16:38.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:37 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:16:38.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:37 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:16:38.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:37 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:16:38.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:37 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:16:38.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:37 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:16:38.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:37 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:16:39.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:39 smithi131 ceph-mon[41654]: pgmap v1470: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:39.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:39 smithi131 ceph-mon[29507]: pgmap v1470: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:39.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:39 smithi157 ceph-mon[33345]: pgmap v1470: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:39.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:16:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:16:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:41 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:41.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:41 smithi131 ceph-mon[41654]: pgmap v1471: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:41 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:41.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:41 smithi131 ceph-mon[29507]: pgmap v1471: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:41 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:41 smithi157 ceph-mon[33345]: pgmap v1471: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:43 smithi131 ceph-mon[41654]: pgmap v1472: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:16:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:16:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:16:43.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:43 smithi131 ceph-mon[29507]: pgmap v1472: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:43.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:43 smithi157 ceph-mon[33345]: pgmap v1472: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:45.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:45 smithi131 ceph-mon[41654]: pgmap v1473: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:45.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:45 smithi131 ceph-mon[29507]: pgmap v1473: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:45.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:45 smithi157 ceph-mon[33345]: pgmap v1473: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:46.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:46 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:16:46.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:46 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:16:46.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:46 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:16:47.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:47 smithi157 ceph-mon[33345]: pgmap v1474: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:47.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:47 smithi131 ceph-mon[41654]: pgmap v1474: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:47.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:47 smithi131 ceph-mon[29507]: pgmap v1474: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:49.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:49 smithi157 ceph-mon[33345]: pgmap v1475: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:49.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:49 smithi131 ceph-mon[41654]: pgmap v1475: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:49.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:49 smithi131 ceph-mon[29507]: pgmap v1475: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:49.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:16:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:16:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:51 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:51 smithi157 ceph-mon[33345]: pgmap v1476: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:51.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:51 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:51.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:51 smithi131 ceph-mon[41654]: pgmap v1476: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:51 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:16:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:51 smithi131 ceph-mon[29507]: pgmap v1476: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:53 smithi131 ceph-mon[41654]: pgmap v1477: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:53.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:53 smithi131 ceph-mon[29507]: pgmap v1477: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:16:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:16:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:16:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:53 smithi157 ceph-mon[33345]: pgmap v1477: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:55 smithi157 ceph-mon[33345]: pgmap v1478: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:55.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:55 smithi131 ceph-mon[29507]: pgmap v1478: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:55 smithi131 ceph-mon[41654]: pgmap v1478: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:16:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:57 smithi157 ceph-mon[33345]: pgmap v1479: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:57.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:57 smithi131 ceph-mon[41654]: pgmap v1479: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:57 smithi131 ceph-mon[29507]: pgmap v1479: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:16:57.834 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.834 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.834 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.834 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.834 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.835 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.835 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.835 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.835 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.835 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.835 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.835 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.835 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.835 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.835 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.835 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.835 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.835 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.835 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.835 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.835 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.836 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.836 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.836 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.836 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.836 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.836 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.836 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.836 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.836 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.836 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.836 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.836 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.836 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.836 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.837 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.837 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.837 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.837 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.837 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.837 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.837 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.837 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.837 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.837 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.837 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.837 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.837 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.837 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.837 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.837 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.841 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:57.841 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:16:59.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:16:59 smithi157 ceph-mon[33345]: pgmap v1480: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:59.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:16:59 smithi131 ceph-mon[29507]: pgmap v1480: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:59.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:16:59 smithi131 ceph-mon[41654]: pgmap v1480: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:16:59.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:16:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:17:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:01 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:01 smithi157 ceph-mon[33345]: pgmap v1481: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:01 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:17:01.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:01 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:01 smithi131 ceph-mon[41654]: pgmap v1481: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:01 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:17:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:01 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:01 smithi131 ceph-mon[29507]: pgmap v1481: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:01 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:17:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:03 smithi131 ceph-mon[41654]: pgmap v1482: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:03 smithi131 ceph-mon[29507]: pgmap v1482: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:17:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:17:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:17:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:03 smithi157 ceph-mon[33345]: pgmap v1482: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:05 smithi157 ceph-mon[33345]: pgmap v1483: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:05.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:05 smithi131 ceph-mon[41654]: pgmap v1483: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:05.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:05 smithi131 ceph-mon[29507]: pgmap v1483: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:07 smithi157 ceph-mon[33345]: pgmap v1484: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:07.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:07 smithi131 ceph-mon[41654]: pgmap v1484: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:07.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:07 smithi131 ceph-mon[29507]: pgmap v1484: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:09.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:09 smithi157 ceph-mon[33345]: pgmap v1485: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:09.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:09 smithi131 ceph-mon[41654]: pgmap v1485: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:09.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:09 smithi131 ceph-mon[29507]: pgmap v1485: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:09.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:17:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:17:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:11 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:11 smithi157 ceph-mon[33345]: pgmap v1486: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:11 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:11 smithi131 ceph-mon[41654]: pgmap v1486: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:11 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:11 smithi131 ceph-mon[29507]: pgmap v1486: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:13 smithi131 ceph-mon[41654]: pgmap v1487: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:17:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:17:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:17:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:13 smithi131 ceph-mon[29507]: pgmap v1487: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:13.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:13 smithi157 ceph-mon[33345]: pgmap v1487: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:15 smithi157 ceph-mon[33345]: pgmap v1488: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:15.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:15 smithi131 ceph-mon[41654]: pgmap v1488: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:15.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:15 smithi131 ceph-mon[29507]: pgmap v1488: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:16.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:17:16.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:17:16.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:17:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:17 smithi157 ceph-mon[33345]: pgmap v1489: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:17.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:17 smithi131 ceph-mon[41654]: pgmap v1489: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:17 smithi131 ceph-mon[29507]: pgmap v1489: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:19.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:19 smithi157 ceph-mon[33345]: pgmap v1490: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:19.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:19 smithi131 ceph-mon[41654]: pgmap v1490: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:19.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:19 smithi131 ceph-mon[29507]: pgmap v1490: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:20.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:17:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:17:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:21 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:21 smithi157 ceph-mon[33345]: pgmap v1491: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:21.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:21 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:21 smithi131 ceph-mon[29507]: pgmap v1491: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:21 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:21 smithi131 ceph-mon[41654]: pgmap v1491: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:23.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:23 smithi131 ceph-mon[29507]: pgmap v1492: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:23 smithi131 ceph-mon[41654]: pgmap v1492: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:17:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:17:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:17:23.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:23 smithi157 ceph-mon[33345]: pgmap v1492: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:25.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:25 smithi157 ceph-mon[33345]: pgmap v1493: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:25.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:25 smithi131 ceph-mon[29507]: pgmap v1493: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:25 smithi131 ceph-mon[41654]: pgmap v1493: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:27 smithi157 ceph-mon[33345]: pgmap v1494: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:27.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:27 smithi131 ceph-mon[29507]: pgmap v1494: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:27.520 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:27 smithi131 ceph-mon[41654]: pgmap v1494: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:29.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:29 smithi157 ceph-mon[33345]: pgmap v1495: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:17:29.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:29 smithi131 ceph-mon[41654]: pgmap v1495: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:17:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:29 smithi131 ceph-mon[29507]: pgmap v1495: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:17:30.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:17:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:17:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:31 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:31 smithi157 ceph-mon[33345]: pgmap v1496: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:31 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:17:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:31 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:31 smithi131 ceph-mon[41654]: pgmap v1496: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:31 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:17:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:31 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:31 smithi131 ceph-mon[29507]: pgmap v1496: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:31 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:17:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:33 smithi131 ceph-mon[41654]: pgmap v1497: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:33.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:33 smithi131 ceph-mon[29507]: pgmap v1497: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:17:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:17:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:17:33.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:33 smithi157 ceph-mon[33345]: pgmap v1497: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:35 smithi157 ceph-mon[33345]: pgmap v1498: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:35.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:35 smithi131 ceph-mon[41654]: pgmap v1498: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:35.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:35 smithi131 ceph-mon[29507]: pgmap v1498: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:37 smithi157 ceph-mon[33345]: pgmap v1499: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:37.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:37 smithi131 ceph-mon[41654]: pgmap v1499: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:37.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:37 smithi131 ceph-mon[29507]: pgmap v1499: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:38.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:38 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:17:38.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:38 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:17:38.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:38 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:17:38.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:38 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:17:38.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:38 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:17:38.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:38 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:17:38.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:38 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:17:38.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:38 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:17:38.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:38 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:17:38.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:38 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:17:38.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:38 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:17:38.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:38 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:17:39.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:39 smithi157 ceph-mon[33345]: pgmap v1500: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:17:39.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:39 smithi131 ceph-mon[29507]: pgmap v1500: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:17:39.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:39 smithi131 ceph-mon[41654]: pgmap v1500: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:17:40.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:17:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:17:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:41 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:41 smithi157 ceph-mon[33345]: pgmap v1501: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:41.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:41 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:41 smithi131 ceph-mon[29507]: pgmap v1501: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:41 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:41 smithi131 ceph-mon[41654]: pgmap v1501: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:43 smithi131 ceph-mon[41654]: pgmap v1502: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:43.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:43 smithi131 ceph-mon[29507]: pgmap v1502: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:17:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:17:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:17:43.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:43 smithi157 ceph-mon[33345]: pgmap v1502: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:45.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:45 smithi157 ceph-mon[33345]: pgmap v1503: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:45 smithi131 ceph-mon[41654]: pgmap v1503: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:45.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:45 smithi131 ceph-mon[29507]: pgmap v1503: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:46.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:46 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:17:46.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:46 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:17:46.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:46 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:17:47.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:47 smithi157 ceph-mon[33345]: pgmap v1504: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:17:47.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:47 smithi131 ceph-mon[41654]: pgmap v1504: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:17:47.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:47 smithi131 ceph-mon[29507]: pgmap v1504: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:17:49.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:49 smithi157 ceph-mon[33345]: pgmap v1505: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:17:49.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:49 smithi131 ceph-mon[41654]: pgmap v1505: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:17:49.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:49 smithi131 ceph-mon[29507]: pgmap v1505: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:17:50.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:17:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:17:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:51 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:51 smithi157 ceph-mon[33345]: pgmap v1506: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:51.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:51 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:51.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:51 smithi131 ceph-mon[41654]: pgmap v1506: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:51 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:17:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:51 smithi131 ceph-mon[29507]: pgmap v1506: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:53 smithi131 ceph-mon[41654]: pgmap v1507: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:53.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:53 smithi131 ceph-mon[29507]: pgmap v1507: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:17:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:17:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:17:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:53 smithi157 ceph-mon[33345]: pgmap v1507: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:55 smithi157 ceph-mon[33345]: pgmap v1508: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:55.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:55 smithi131 ceph-mon[41654]: pgmap v1508: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:55.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:55 smithi131 ceph-mon[29507]: pgmap v1508: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:17:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:57 smithi157 ceph-mon[33345]: pgmap v1509: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:57.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:57 smithi131 ceph-mon[41654]: pgmap v1509: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:57 smithi131 ceph-mon[29507]: pgmap v1509: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:17:59.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:17:59 smithi157 ceph-mon[33345]: pgmap v1510: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:17:59.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:17:59 smithi131 ceph-mon[41654]: pgmap v1510: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:17:59.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:17:59 smithi131 ceph-mon[29507]: pgmap v1510: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:00.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:17:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:18:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:01 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:01 smithi157 ceph-mon[33345]: pgmap v1511: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:01 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:18:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:01 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:01 smithi131 ceph-mon[41654]: pgmap v1511: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:01 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:18:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:01 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:01 smithi131 ceph-mon[29507]: pgmap v1511: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:01 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:18:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:03 smithi131 ceph-mon[41654]: pgmap v1512: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:03 smithi131 ceph-mon[29507]: pgmap v1512: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:18:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:18:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:18:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:03 smithi157 ceph-mon[33345]: pgmap v1512: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:05 smithi157 ceph-mon[33345]: pgmap v1513: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:05.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:05 smithi131 ceph-mon[41654]: pgmap v1513: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:05.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:05 smithi131 ceph-mon[29507]: pgmap v1513: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:07 smithi157 ceph-mon[33345]: pgmap v1514: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:07.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:07 smithi131 ceph-mon[41654]: pgmap v1514: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:07.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:07 smithi131 ceph-mon[29507]: pgmap v1514: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:09.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:09 smithi157 ceph-mon[33345]: pgmap v1515: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:09.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:09 smithi131 ceph-mon[41654]: pgmap v1515: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:09.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:09 smithi131 ceph-mon[29507]: pgmap v1515: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:10.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:18:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:18:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:11 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:11 smithi157 ceph-mon[33345]: pgmap v1516: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:11.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:11 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:11 smithi131 ceph-mon[41654]: pgmap v1516: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:11 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:11 smithi131 ceph-mon[29507]: pgmap v1516: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:13.196 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:18:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:18:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:18:13.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:13 smithi157 ceph-mon[33345]: pgmap v1517: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:13.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:13 smithi131 ceph-mon[29507]: pgmap v1517: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:13.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:13 smithi131 ceph-mon[41654]: pgmap v1517: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:15 smithi157 ceph-mon[33345]: pgmap v1518: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:15.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:15 smithi131 ceph-mon[29507]: pgmap v1518: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:15.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:15 smithi131 ceph-mon[41654]: pgmap v1518: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:16.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:18:16.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:18:16.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:18:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:17 smithi157 ceph-mon[33345]: pgmap v1519: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:17.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:17 smithi131 ceph-mon[41654]: pgmap v1519: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:17 smithi131 ceph-mon[29507]: pgmap v1519: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:19.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:19 smithi157 ceph-mon[33345]: pgmap v1520: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:19.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:19 smithi131 ceph-mon[41654]: pgmap v1520: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:19.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:19 smithi131 ceph-mon[29507]: pgmap v1520: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:20.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:18:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:18:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:21 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:21 smithi157 ceph-mon[33345]: pgmap v1521: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:21.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:21 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:21.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:21 smithi131 ceph-mon[41654]: pgmap v1521: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:21.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:21 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:21.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:21 smithi131 ceph-mon[29507]: pgmap v1521: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:23.217 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:18:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:18:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:18:23.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:23 smithi157 ceph-mon[33345]: pgmap v1522: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:23.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:23 smithi131 ceph-mon[29507]: pgmap v1522: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:23.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:23 smithi131 ceph-mon[41654]: pgmap v1522: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:25 smithi131 ceph-mon[41654]: pgmap v1523: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:25.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:25 smithi131 ceph-mon[29507]: pgmap v1523: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:25.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:25 smithi157 ceph-mon[33345]: pgmap v1523: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:27.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:27 smithi131 ceph-mon[41654]: pgmap v1524: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:27.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:27 smithi131 ceph-mon[29507]: pgmap v1524: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:27.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:27 smithi157 ceph-mon[33345]: pgmap v1524: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:29 smithi131 ceph-mon[41654]: pgmap v1525: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:29.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:29 smithi131 ceph-mon[29507]: pgmap v1525: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:29.522 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:29 smithi157 ceph-mon[33345]: pgmap v1525: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:30.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:18:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:18:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:31 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:31 smithi131 ceph-mon[41654]: pgmap v1526: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:31 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:18:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:31 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:31 smithi131 ceph-mon[29507]: pgmap v1526: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:31 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:18:31.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:31 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:31.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:31 smithi157 ceph-mon[33345]: pgmap v1526: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:31.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:31 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:18:33.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:32 smithi157 ceph-mon[33345]: pgmap v1527: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:33.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:32 smithi131 ceph-mon[41654]: pgmap v1527: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:33.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:32 smithi131 ceph-mon[29507]: pgmap v1527: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:18:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:18:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:18:35.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:34 smithi157 ceph-mon[33345]: pgmap v1528: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:35.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:34 smithi131 ceph-mon[41654]: pgmap v1528: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:35.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:34 smithi131 ceph-mon[29507]: pgmap v1528: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:37.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:36 smithi157 ceph-mon[33345]: pgmap v1529: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:37.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:36 smithi131 ceph-mon[41654]: pgmap v1529: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:37.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:36 smithi131 ceph-mon[29507]: pgmap v1529: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:37.845 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.846 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.846 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.846 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.846 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.846 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.846 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.846 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.846 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.846 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.846 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.846 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.846 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.846 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.846 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.847 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.847 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.847 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.847 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.847 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.847 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.847 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.847 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.847 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.847 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.847 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.847 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.847 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.847 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.848 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.848 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.848 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.848 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.848 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.848 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.848 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.848 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.848 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.848 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.848 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.848 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.848 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.848 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.848 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.849 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.849 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.849 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.849 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.849 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.849 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.849 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.849 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.849 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.849 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.849 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.849 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.849 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.849 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.852 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.852 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.852 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.852 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.852 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.852 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.852 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.852 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:37.852 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:18:39.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:38 smithi157 ceph-mon[33345]: pgmap v1530: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:39.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:38 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:18:39.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:38 smithi131 ceph-mon[41654]: pgmap v1530: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:39.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:38 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:18:39.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:38 smithi131 ceph-mon[29507]: pgmap v1530: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:39.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:38 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:18:40.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:18:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:18:41.108 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:40 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:41.108 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:40 smithi131 ceph-mon[41654]: pgmap v1531: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:41.109 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:40 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:41.109 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:40 smithi131 ceph-mon[29507]: pgmap v1531: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:40 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:41.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:40 smithi157 ceph-mon[33345]: pgmap v1531: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:43.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:42 smithi157 ceph-mon[33345]: pgmap v1532: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:43.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:42 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:18:43.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:42 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:18:43.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:42 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:18:43.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:42 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:18:43.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:42 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:18:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:18:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:18:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:18:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:42 smithi131 ceph-mon[41654]: pgmap v1532: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:42 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:18:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:42 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:18:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:42 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:18:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:42 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:18:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:42 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:18:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:42 smithi131 ceph-mon[29507]: pgmap v1532: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:42 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:18:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:42 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:18:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:42 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:18:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:42 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:18:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:42 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:18:45.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:44 smithi157 ceph-mon[33345]: pgmap v1533: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:45.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:44 smithi131 ceph-mon[29507]: pgmap v1533: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:45.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:44 smithi131 ceph-mon[41654]: pgmap v1533: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:46.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:45 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:18:46.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:45 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:18:46.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:45 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:18:47.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:46 smithi157 ceph-mon[33345]: pgmap v1534: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:47.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:46 smithi131 ceph-mon[41654]: pgmap v1534: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:47.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:46 smithi131 ceph-mon[29507]: pgmap v1534: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:18:49.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:48 smithi157 ceph-mon[33345]: pgmap v1535: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:49.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:48 smithi131 ceph-mon[29507]: pgmap v1535: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:49.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:48 smithi131 ceph-mon[41654]: pgmap v1535: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:50.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:18:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:18:51.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:50 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:51.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:50 smithi157 ceph-mon[33345]: pgmap v1536: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:50 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:51.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:50 smithi131 ceph-mon[41654]: pgmap v1536: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:50 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:18:51.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:50 smithi131 ceph-mon[29507]: pgmap v1536: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:53.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:52 smithi157 ceph-mon[33345]: pgmap v1537: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:52 smithi131 ceph-mon[41654]: pgmap v1537: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:18:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:18:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:18:53.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:52 smithi131 ceph-mon[29507]: pgmap v1537: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:54 smithi131 ceph-mon[41654]: pgmap v1538: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:55.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:54 smithi131 ceph-mon[29507]: pgmap v1538: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:54 smithi157 ceph-mon[33345]: pgmap v1538: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:18:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:56 smithi131 ceph-mon[41654]: pgmap v1539: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:56 smithi131 ceph-mon[29507]: pgmap v1539: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:56 smithi157 ceph-mon[33345]: pgmap v1539: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:59.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:18:58 smithi131 ceph-mon[29507]: pgmap v1540: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:59.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:18:58 smithi131 ceph-mon[41654]: pgmap v1540: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:18:59.272 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:18:58 smithi157 ceph-mon[33345]: pgmap v1540: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:00.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:18:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:19:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:00 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:00 smithi131 ceph-mon[29507]: pgmap v1541: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:19:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:00 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:00 smithi131 ceph-mon[41654]: pgmap v1541: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:19:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:00 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:00 smithi157 ceph-mon[33345]: pgmap v1541: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:19:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:02 smithi131 ceph-mon[29507]: pgmap v1542: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:03.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:02 smithi131 ceph-mon[41654]: pgmap v1542: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:03.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:19:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:19:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:19:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:02 smithi157 ceph-mon[33345]: pgmap v1542: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:04 smithi131 ceph-mon[41654]: pgmap v1543: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:05.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:04 smithi131 ceph-mon[29507]: pgmap v1543: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:04 smithi157 ceph-mon[33345]: pgmap v1543: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:07.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:06 smithi131 ceph-mon[41654]: pgmap v1544: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:07.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:06 smithi131 ceph-mon[29507]: pgmap v1544: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:06 smithi157 ceph-mon[33345]: pgmap v1544: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:09.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:09 smithi131 ceph-mon[41654]: pgmap v1545: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:09.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:09 smithi131 ceph-mon[29507]: pgmap v1545: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:09.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:09 smithi157 ceph-mon[33345]: pgmap v1545: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:10.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:19:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:19:11.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:11 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:11 smithi131 ceph-mon[41654]: pgmap v1546: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:11 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:11 smithi131 ceph-mon[29507]: pgmap v1546: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:11 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:11 smithi157 ceph-mon[33345]: pgmap v1546: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:13 smithi131 ceph-mon[41654]: pgmap v1547: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:13 smithi131 ceph-mon[29507]: pgmap v1547: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:19:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:19:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:19:13.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:13 smithi157 ceph-mon[33345]: pgmap v1547: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:15.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:15 smithi131 ceph-mon[41654]: pgmap v1548: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:15.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:15 smithi131 ceph-mon[29507]: pgmap v1548: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:15 smithi157 ceph-mon[33345]: pgmap v1548: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:16.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:19:16.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:19:16.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:19:17.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:17 smithi131 ceph-mon[41654]: pgmap v1549: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:17 smithi131 ceph-mon[29507]: pgmap v1549: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:17 smithi157 ceph-mon[33345]: pgmap v1549: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:19.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:19 smithi157 ceph-mon[33345]: pgmap v1550: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:19.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:19 smithi131 ceph-mon[41654]: pgmap v1550: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:19.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:19 smithi131 ceph-mon[29507]: pgmap v1550: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:20.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:19:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:19:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:21 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:21 smithi157 ceph-mon[33345]: pgmap v1551: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:21 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:21 smithi131 ceph-mon[41654]: pgmap v1551: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:21 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:21 smithi131 ceph-mon[29507]: pgmap v1551: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:19:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:19:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:19:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:23 smithi131 ceph-mon[41654]: pgmap v1552: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:23.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:23 smithi131 ceph-mon[29507]: pgmap v1552: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:23.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:23 smithi157 ceph-mon[33345]: pgmap v1552: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:25.469 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:25 smithi131 ceph-mon[41654]: pgmap v1553: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:25.469 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:25 smithi131 ceph-mon[29507]: pgmap v1553: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:25.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:25 smithi157 ceph-mon[33345]: pgmap v1553: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:27 smithi157 ceph-mon[33345]: pgmap v1554: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:27.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:27 smithi131 ceph-mon[41654]: pgmap v1554: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:27.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:27 smithi131 ceph-mon[29507]: pgmap v1554: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:29.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:29 smithi157 ceph-mon[33345]: pgmap v1555: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:29 smithi131 ceph-mon[41654]: pgmap v1555: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:29 smithi131 ceph-mon[29507]: pgmap v1555: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:30.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:19:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:19:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:31 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:31 smithi157 ceph-mon[33345]: pgmap v1556: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:31 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:19:31.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:31 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:31 smithi131 ceph-mon[41654]: pgmap v1556: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:31 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:19:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:31 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:31 smithi131 ceph-mon[29507]: pgmap v1556: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:31 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:19:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:33 smithi131 ceph-mon[41654]: pgmap v1557: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:19:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:19:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:19:33.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:33 smithi131 ceph-mon[29507]: pgmap v1557: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:33.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:33 smithi157 ceph-mon[33345]: pgmap v1557: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:35.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:35 smithi131 ceph-mon[41654]: pgmap v1558: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:35.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:35 smithi131 ceph-mon[29507]: pgmap v1558: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:35 smithi157 ceph-mon[33345]: pgmap v1558: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:37 smithi157 ceph-mon[33345]: pgmap v1559: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:37.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:37 smithi131 ceph-mon[41654]: pgmap v1559: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:37.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:37 smithi131 ceph-mon[29507]: pgmap v1559: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:39.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:39 smithi157 ceph-mon[33345]: pgmap v1560: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:39.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:39 smithi131 ceph-mon[29507]: pgmap v1560: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:39.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:39 smithi131 ceph-mon[41654]: pgmap v1560: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:40.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:19:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:19:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:41 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:41 smithi157 ceph-mon[33345]: pgmap v1561: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:41.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:41 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:41 smithi131 ceph-mon[41654]: pgmap v1561: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:41 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:41 smithi131 ceph-mon[29507]: pgmap v1561: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:43.266 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:19:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:19:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:19:43.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:43 smithi131 ceph-mon[29507]: pgmap v1562: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:43.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:43 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:19:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:43 smithi131 ceph-mon[41654]: pgmap v1562: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:43 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:19:43.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:43 smithi157 ceph-mon[33345]: pgmap v1562: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:43.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:43 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:19:45.201 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:45 smithi157 ceph-mon[33345]: pgmap v1563: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:45 smithi131 ceph-mon[41654]: pgmap v1563: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:45.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:45 smithi131 ceph-mon[29507]: pgmap v1563: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:46.409 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:46 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:19:46.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:46 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:19:46.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:46 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:19:47.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:47 smithi157 ceph-mon[33345]: pgmap v1564: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:47.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:47 smithi131 ceph-mon[41654]: pgmap v1564: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:47.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:47 smithi131 ceph-mon[29507]: pgmap v1564: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:19:48.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:48 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:19:48.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:48 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:19:48.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:48 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:19:48.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:48 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:19:48.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:48 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:19:48.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:48 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:19:48.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:48 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:19:48.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:48 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:19:48.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:48 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:19:48.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:48 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:19:48.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:48 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:19:48.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:48 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:19:48.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:48 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:19:48.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:48 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:19:48.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:48 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:19:49.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:49 smithi157 ceph-mon[33345]: pgmap v1565: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:49.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:49 smithi131 ceph-mon[41654]: pgmap v1565: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:49.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:49 smithi131 ceph-mon[29507]: pgmap v1565: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:50.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:19:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:19:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:51 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:51 smithi157 ceph-mon[33345]: pgmap v1566: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:51.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:51 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:51.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:51 smithi131 ceph-mon[41654]: pgmap v1566: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:51 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:19:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:51 smithi131 ceph-mon[29507]: pgmap v1566: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:53.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:53 smithi131 ceph-mon[41654]: pgmap v1567: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:53.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:53 smithi131 ceph-mon[29507]: pgmap v1567: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:19:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:19:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:19:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:53 smithi157 ceph-mon[33345]: pgmap v1567: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:55 smithi157 ceph-mon[33345]: pgmap v1568: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:55.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:55 smithi131 ceph-mon[41654]: pgmap v1568: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:55.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:55 smithi131 ceph-mon[29507]: pgmap v1568: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:19:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:57 smithi157 ceph-mon[33345]: pgmap v1569: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:57.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:57 smithi131 ceph-mon[29507]: pgmap v1569: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:57 smithi131 ceph-mon[41654]: pgmap v1569: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:59.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:19:59 smithi157 ceph-mon[33345]: pgmap v1570: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:59.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:19:59 smithi131 ceph-mon[41654]: pgmap v1570: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:19:59.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:19:59 smithi131 ceph-mon[29507]: pgmap v1570: 228 pgs: 228 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:00.157 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:19:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:20:00.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:00 smithi157 ceph-mon[33345]: overall HEALTH_WARN 3 pool(s) do not have an application enabled 2024-02-05T08:20:00.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:00 smithi131 ceph-mon[29507]: overall HEALTH_WARN 3 pool(s) do not have an application enabled 2024-02-05T08:20:00.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:00 smithi131 ceph-mon[41654]: overall HEALTH_WARN 3 pool(s) do not have an application enabled 2024-02-05T08:20:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:01 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:01 smithi157 ceph-mon[33345]: pgmap v1571: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:01 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:20:01.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:01 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:01 smithi131 ceph-mon[29507]: pgmap v1571: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:01 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:20:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:01 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:01 smithi131 ceph-mon[41654]: pgmap v1571: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:01 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:20:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:03 smithi131 ceph-mon[29507]: pgmap v1572: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:03 smithi131 ceph-mon[41654]: pgmap v1572: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:20:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:20:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:20:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:03 smithi157 ceph-mon[33345]: pgmap v1572: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:05 smithi157 ceph-mon[33345]: pgmap v1573: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:05.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:05 smithi131 ceph-mon[29507]: pgmap v1573: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:05.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:05 smithi131 ceph-mon[41654]: pgmap v1573: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:07 smithi157 ceph-mon[33345]: pgmap v1574: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:07.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:07 smithi131 ceph-mon[29507]: pgmap v1574: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:07.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:07 smithi131 ceph-mon[41654]: pgmap v1574: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:09.220 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:09 smithi157 ceph-mon[33345]: pgmap v1575: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:09.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:09 smithi131 ceph-mon[29507]: pgmap v1575: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:09.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:09 smithi131 ceph-mon[41654]: pgmap v1575: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:10.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:20:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:20:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:11 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:11 smithi157 ceph-mon[33345]: pgmap v1576: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:11 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:11 smithi131 ceph-mon[41654]: pgmap v1576: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:11 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:11 smithi131 ceph-mon[29507]: pgmap v1576: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:20:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:13 smithi131 ceph-mon[41654]: pgmap v1577: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:13 smithi131 ceph-mon[29507]: pgmap v1577: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:20:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:20:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:20:13.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:13 smithi157 ceph-mon[33345]: pgmap v1577: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:15 smithi157 ceph-mon[33345]: pgmap v1578: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:15.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:15 smithi131 ceph-mon[29507]: pgmap v1578: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:15.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:15 smithi131 ceph-mon[41654]: pgmap v1578: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:16.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:20:16.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:20:16.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:20:16.858 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.858 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.858 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.858 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.858 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.858 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.858 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.858 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.858 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.858 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.858 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.858 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.858 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.859 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.860 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.860 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.860 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.860 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.860 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.860 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.860 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.860 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.860 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.860 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.860 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.860 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.860 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.860 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.861 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.861 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.861 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.861 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.861 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.861 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.861 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.861 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.861 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.861 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.861 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.861 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.861 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.861 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.861 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.861 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.862 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.863 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.863 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.863 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.863 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.863 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.863 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.863 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.863 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.863 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.863 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.863 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.863 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.863 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.863 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.863 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.863 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.864 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.864 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.864 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.864 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.864 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.864 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.864 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:16.864 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:20:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:17 smithi157 ceph-mon[33345]: pgmap v1579: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:17.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:17 smithi131 ceph-mon[41654]: pgmap v1579: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:17 smithi131 ceph-mon[29507]: pgmap v1579: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:19.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:19 smithi157 ceph-mon[33345]: pgmap v1580: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:19.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:19 smithi131 ceph-mon[41654]: pgmap v1580: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:19.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:19 smithi131 ceph-mon[29507]: pgmap v1580: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:20.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:20:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:20:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:21 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:21 smithi157 ceph-mon[33345]: pgmap v1581: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:21.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:21 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:21 smithi131 ceph-mon[41654]: pgmap v1581: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:21 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:21 smithi131 ceph-mon[29507]: pgmap v1581: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:23.205 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:20:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:20:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:20:23.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:23 smithi157 ceph-mon[33345]: pgmap v1582: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:23.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:23 smithi131 ceph-mon[41654]: pgmap v1582: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:23.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:23 smithi131 ceph-mon[29507]: pgmap v1582: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:25.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:25 smithi157 ceph-mon[33345]: pgmap v1583: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:25.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:25 smithi131 ceph-mon[41654]: pgmap v1583: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:25.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:25 smithi131 ceph-mon[29507]: pgmap v1583: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:27 smithi157 ceph-mon[33345]: pgmap v1584: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:27.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:27 smithi131 ceph-mon[41654]: pgmap v1584: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:27.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:27 smithi131 ceph-mon[29507]: pgmap v1584: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:29.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:29 smithi157 ceph-mon[33345]: pgmap v1585: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:29.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:29 smithi131 ceph-mon[41654]: pgmap v1585: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:29 smithi131 ceph-mon[29507]: pgmap v1585: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:30.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:20:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:20:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:31 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:31 smithi131 ceph-mon[29507]: pgmap v1586: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:31 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:20:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:31 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:31 smithi131 ceph-mon[41654]: pgmap v1586: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:31 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:20:31.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:31 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:31.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:31 smithi157 ceph-mon[33345]: pgmap v1586: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:31.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:31 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:20:33.223 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:20:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:20:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:20:33.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:33 smithi131 ceph-mon[29507]: pgmap v1587: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:33.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:33 smithi131 ceph-mon[41654]: pgmap v1587: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:33.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:33 smithi157 ceph-mon[33345]: pgmap v1587: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:35.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:35 smithi131 ceph-mon[41654]: pgmap v1588: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:35.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:35 smithi131 ceph-mon[29507]: pgmap v1588: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:35.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:35 smithi157 ceph-mon[33345]: pgmap v1588: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:37.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:37 smithi131 ceph-mon[41654]: pgmap v1589: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:37.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:37 smithi131 ceph-mon[29507]: pgmap v1589: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:37.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:37 smithi157 ceph-mon[33345]: pgmap v1589: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:39.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:39 smithi131 ceph-mon[41654]: pgmap v1590: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:39.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:39 smithi131 ceph-mon[29507]: pgmap v1590: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:39.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:39 smithi157 ceph-mon[33345]: pgmap v1590: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:40.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:20:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:20:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:41 smithi131 ceph-mon[41654]: pgmap v1591: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:41 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:41 smithi131 ceph-mon[29507]: pgmap v1591: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:41 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:41.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:41 smithi157 ceph-mon[33345]: pgmap v1591: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:41.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:41 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:43.237 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:20:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:20:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:20:43.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:43 smithi131 ceph-mon[41654]: pgmap v1592: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:43.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:43 smithi131 ceph-mon[29507]: pgmap v1592: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:43.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:43 smithi157 ceph-mon[33345]: pgmap v1592: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:45 smithi131 ceph-mon[41654]: pgmap v1593: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:45.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:45 smithi131 ceph-mon[29507]: pgmap v1593: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:45.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:45 smithi157 ceph-mon[33345]: pgmap v1593: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:46.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:46 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:20:46.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:46 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:20:46.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:46 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:20:47.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:47 smithi131 ceph-mon[41654]: pgmap v1594: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:47.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:47 smithi131 ceph-mon[29507]: pgmap v1594: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:47.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:47 smithi157 ceph-mon[33345]: pgmap v1594: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:48.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:48 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:20:48.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:48 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:20:48.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:48 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:20:49.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:49 smithi131 ceph-mon[41654]: pgmap v1595: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:49.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:49 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:20:49.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:49 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:20:49.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:49 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:20:49.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:49 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:20:49.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:49 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:20:49.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:49 smithi131 ceph-mon[29507]: pgmap v1595: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:49.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:49 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:20:49.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:49 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:20:49.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:49 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:20:49.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:49 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:20:49.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:49 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:20:49.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:49 smithi157 ceph-mon[33345]: pgmap v1595: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:49.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:49 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:20:49.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:49 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:20:49.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:49 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:20:49.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:49 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:20:49.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:49 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:20:50.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:20:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:20:50.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:50 smithi131 ceph-mon[29507]: pgmap v1596: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:50.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:50 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:50.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:50 smithi131 ceph-mon[41654]: pgmap v1596: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:50.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:50 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:50.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:50 smithi157 ceph-mon[33345]: pgmap v1596: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:50.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:50 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:20:53.220 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:52 smithi131 ceph-mon[41654]: pgmap v1597: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:53.220 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:52 smithi131 ceph-mon[29507]: pgmap v1597: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:53.221 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:20:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:20:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:20:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:52 smithi157 ceph-mon[33345]: pgmap v1597: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:54 smithi131 ceph-mon[41654]: pgmap v1598: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:55.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:54 smithi131 ceph-mon[29507]: pgmap v1598: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:54 smithi157 ceph-mon[33345]: pgmap v1598: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:20:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:56 smithi131 ceph-mon[41654]: pgmap v1599: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:56 smithi131 ceph-mon[29507]: pgmap v1599: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:56 smithi157 ceph-mon[33345]: pgmap v1599: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:20:59.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:20:58 smithi131 ceph-mon[29507]: pgmap v1600: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:59.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:20:58 smithi131 ceph-mon[41654]: pgmap v1600: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:20:59.272 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:20:58 smithi157 ceph-mon[33345]: pgmap v1600: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:00.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:20:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:21:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:00 smithi131 ceph-mon[41654]: pgmap v1601: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:00 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:00 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:21:01.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:00 smithi131 ceph-mon[29507]: pgmap v1601: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:00 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:00 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:21:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:00 smithi157 ceph-mon[33345]: pgmap v1601: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:00 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:00 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:21:03.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:02 smithi131 ceph-mon[41654]: pgmap v1602: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:02 smithi131 ceph-mon[29507]: pgmap v1602: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:21:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:21:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:21:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:02 smithi157 ceph-mon[33345]: pgmap v1602: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:05.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:04 smithi131 ceph-mon[29507]: pgmap v1603: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:05.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:04 smithi131 ceph-mon[41654]: pgmap v1603: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:04 smithi157 ceph-mon[33345]: pgmap v1603: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:07.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:07 smithi131 ceph-mon[41654]: pgmap v1604: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:07.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:07 smithi131 ceph-mon[29507]: pgmap v1604: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:07 smithi157 ceph-mon[33345]: pgmap v1604: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:09.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:09 smithi131 ceph-mon[41654]: pgmap v1605: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:09.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:09 smithi131 ceph-mon[29507]: pgmap v1605: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:09.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:09 smithi157 ceph-mon[33345]: pgmap v1605: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:10.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:21:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:21:11.266 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:11 smithi131 ceph-mon[29507]: pgmap v1606: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:11.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:11 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:11 smithi131 ceph-mon[41654]: pgmap v1606: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:11.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:11 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:11 smithi157 ceph-mon[33345]: pgmap v1606: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:11 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:13.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:13 smithi131 ceph-mon[41654]: pgmap v1607: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:21:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:21:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:21:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:13 smithi131 ceph-mon[29507]: pgmap v1607: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:13.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:13 smithi157 ceph-mon[33345]: pgmap v1607: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:15.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:15 smithi131 ceph-mon[29507]: pgmap v1608: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:15.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:15 smithi131 ceph-mon[41654]: pgmap v1608: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:15 smithi157 ceph-mon[33345]: pgmap v1608: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:16.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:21:16.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:21:16.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:21:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:17 smithi157 ceph-mon[33345]: pgmap v1609: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:17.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:17 smithi131 ceph-mon[41654]: pgmap v1609: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:17 smithi131 ceph-mon[29507]: pgmap v1609: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:19.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:19 smithi157 ceph-mon[33345]: pgmap v1610: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:19.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:19 smithi131 ceph-mon[29507]: pgmap v1610: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:19.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:19 smithi131 ceph-mon[41654]: pgmap v1610: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:20.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:21:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:21:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:21 smithi157 ceph-mon[33345]: pgmap v1611: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:21 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:21 smithi131 ceph-mon[29507]: pgmap v1611: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:21 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:21 smithi131 ceph-mon[41654]: pgmap v1611: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:21 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:23 smithi131 ceph-mon[41654]: pgmap v1612: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:23.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:23 smithi131 ceph-mon[29507]: pgmap v1612: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:21:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:21:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:21:23.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:23 smithi157 ceph-mon[33345]: pgmap v1612: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:25.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:25 smithi157 ceph-mon[33345]: pgmap v1613: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:25.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:25 smithi131 ceph-mon[41654]: pgmap v1613: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:25.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:25 smithi131 ceph-mon[29507]: pgmap v1613: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:27 smithi157 ceph-mon[33345]: pgmap v1614: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:21:27.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:27 smithi131 ceph-mon[41654]: pgmap v1614: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:21:27.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:27 smithi131 ceph-mon[29507]: pgmap v1614: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:21:29.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:29 smithi157 ceph-mon[33345]: pgmap v1615: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:21:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:29 smithi131 ceph-mon[41654]: pgmap v1615: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:21:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:29 smithi131 ceph-mon[29507]: pgmap v1615: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:21:29.958 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 08:21:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a[67165]: logger=cleanup t=2024-02-05T08:21:29.496802794Z level=info msg="Completed cleanup jobs" duration=4.597341ms 2024-02-05T08:21:30.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:21:29 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:21:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:31 smithi157 ceph-mon[33345]: pgmap v1616: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:31 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:31 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:21:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:31 smithi131 ceph-mon[41654]: pgmap v1616: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:31 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:31 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:21:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:31 smithi131 ceph-mon[29507]: pgmap v1616: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:31 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:31 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:21:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:33 smithi131 ceph-mon[41654]: pgmap v1617: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:21:33.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:33 smithi131 ceph-mon[29507]: pgmap v1617: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:21:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:21:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:21:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:21:33.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:33 smithi157 ceph-mon[33345]: pgmap v1617: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:21:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:35 smithi157 ceph-mon[33345]: pgmap v1618: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:35.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:35 smithi131 ceph-mon[41654]: pgmap v1618: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:35.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:35 smithi131 ceph-mon[29507]: pgmap v1618: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:37 smithi157 ceph-mon[33345]: pgmap v1619: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:37.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:37 smithi131 ceph-mon[41654]: pgmap v1619: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:37.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:37 smithi131 ceph-mon[29507]: pgmap v1619: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:39.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:39 smithi157 ceph-mon[33345]: pgmap v1620: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:39.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:39 smithi131 ceph-mon[41654]: pgmap v1620: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:39.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:39 smithi131 ceph-mon[29507]: pgmap v1620: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:40.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:21:39 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:21:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:41 smithi157 ceph-mon[33345]: pgmap v1621: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:41 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:41.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:41 smithi131 ceph-mon[41654]: pgmap v1621: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:41 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:41 smithi131 ceph-mon[29507]: pgmap v1621: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:41 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:43 smithi131 ceph-mon[41654]: pgmap v1622: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:43.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:43 smithi131 ceph-mon[29507]: pgmap v1622: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:21:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:21:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:21:43.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:43 smithi157 ceph-mon[33345]: pgmap v1622: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:45.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:45 smithi157 ceph-mon[33345]: pgmap v1623: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:45.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:45 smithi131 ceph-mon[29507]: pgmap v1623: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:45 smithi131 ceph-mon[41654]: pgmap v1623: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:46.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:46 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:21:46.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:46 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:21:46.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:46 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:21:47.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:47 smithi157 ceph-mon[33345]: pgmap v1624: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:47.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:47 smithi131 ceph-mon[41654]: pgmap v1624: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:47.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:47 smithi131 ceph-mon[29507]: pgmap v1624: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:21:49.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:49 smithi157 ceph-mon[33345]: pgmap v1625: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:49.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:49 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:21:49.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:49 smithi131 ceph-mon[41654]: pgmap v1625: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:49.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:49 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:21:49.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:49 smithi131 ceph-mon[29507]: pgmap v1625: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:49.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:49 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:21:50.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:21:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:21:50.970 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:50 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:21:50.970 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:50 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:21:50.970 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:50 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:21:50.970 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:50 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:21:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:50 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:21:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:50 smithi157 ceph-mon[33345]: pgmap v1626: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:50.971 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:50 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:51.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:50 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:21:51.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:50 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:21:51.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:50 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:21:51.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:50 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:21:51.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:50 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:21:51.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:50 smithi131 ceph-mon[29507]: pgmap v1626: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:51.017 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:50 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:51.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:50 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:21:51.017 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:50 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:21:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:50 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:21:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:50 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:21:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:50 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:21:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:50 smithi131 ceph-mon[41654]: pgmap v1626: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:51.018 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:50 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:21:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:53 smithi131 ceph-mon[41654]: pgmap v1627: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:21:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:21:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:21:53.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:53 smithi131 ceph-mon[29507]: pgmap v1627: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:53 smithi157 ceph-mon[33345]: pgmap v1627: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:55.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:55 smithi131 ceph-mon[41654]: pgmap v1628: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:55.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:55 smithi131 ceph-mon[29507]: pgmap v1628: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:55 smithi157 ceph-mon[33345]: pgmap v1628: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:21:56.870 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.870 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.870 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.870 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.870 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.871 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.872 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.873 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.874 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.875 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.876 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:56.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:21:57.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:57 smithi131 ceph-mon[41654]: pgmap v1629: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:57.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:57 smithi131 ceph-mon[29507]: pgmap v1629: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:57 smithi157 ceph-mon[33345]: pgmap v1629: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:59.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:21:59 smithi131 ceph-mon[41654]: pgmap v1630: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:59.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:21:59 smithi131 ceph-mon[29507]: pgmap v1630: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:21:59.272 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:21:59 smithi157 ceph-mon[33345]: pgmap v1630: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:22:00.472 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:21:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:22:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:01 smithi131 ceph-mon[41654]: pgmap v1631: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:01 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:01.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:01 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:22:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:01 smithi131 ceph-mon[29507]: pgmap v1631: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:01 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:01.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:01 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:22:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:01 smithi157 ceph-mon[33345]: pgmap v1631: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:01 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:01 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:22:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:03 smithi131 ceph-mon[41654]: pgmap v1632: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:03 smithi131 ceph-mon[29507]: pgmap v1632: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:22:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:22:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:22:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:03 smithi157 ceph-mon[33345]: pgmap v1632: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:05 smithi157 ceph-mon[33345]: pgmap v1633: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:05.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:05 smithi131 ceph-mon[41654]: pgmap v1633: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:05.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:05 smithi131 ceph-mon[29507]: pgmap v1633: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:07 smithi157 ceph-mon[33345]: pgmap v1634: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:07.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:07 smithi131 ceph-mon[41654]: pgmap v1634: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:07.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:07 smithi131 ceph-mon[29507]: pgmap v1634: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:09.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:09 smithi157 ceph-mon[33345]: pgmap v1635: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:09.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:09 smithi131 ceph-mon[41654]: pgmap v1635: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:09.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:09 smithi131 ceph-mon[29507]: pgmap v1635: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:10.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:22:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:22:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:11 smithi157 ceph-mon[33345]: pgmap v1636: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:11 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:11 smithi131 ceph-mon[41654]: pgmap v1636: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:11 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:11 smithi131 ceph-mon[29507]: pgmap v1636: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:11 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:13 smithi131 ceph-mon[41654]: pgmap v1637: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:22:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:22:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:22:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:13 smithi131 ceph-mon[29507]: pgmap v1637: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:13.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:13 smithi157 ceph-mon[33345]: pgmap v1637: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:15 smithi157 ceph-mon[33345]: pgmap v1638: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:15.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:15 smithi131 ceph-mon[41654]: pgmap v1638: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:15.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:15 smithi131 ceph-mon[29507]: pgmap v1638: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:16.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:22:16.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:22:16.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:22:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:17 smithi157 ceph-mon[33345]: pgmap v1639: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:17.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:17 smithi131 ceph-mon[41654]: pgmap v1639: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:17 smithi131 ceph-mon[29507]: pgmap v1639: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:19.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:19 smithi157 ceph-mon[33345]: pgmap v1640: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:22:19.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:19 smithi131 ceph-mon[41654]: pgmap v1640: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:22:19.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:19 smithi131 ceph-mon[29507]: pgmap v1640: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:22:20.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:22:20 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:22:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:21 smithi157 ceph-mon[33345]: pgmap v1641: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:21 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:21 smithi131 ceph-mon[41654]: pgmap v1641: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:21 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:21 smithi131 ceph-mon[29507]: pgmap v1641: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:21 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:23 smithi131 ceph-mon[41654]: pgmap v1642: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:23.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:23 smithi131 ceph-mon[29507]: pgmap v1642: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:22:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:22:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:22:23.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:23 smithi157 ceph-mon[33345]: pgmap v1642: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:25.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:25 smithi157 ceph-mon[33345]: pgmap v1643: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:25 smithi131 ceph-mon[41654]: pgmap v1643: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:25.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:25 smithi131 ceph-mon[29507]: pgmap v1643: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:27 smithi157 ceph-mon[33345]: pgmap v1644: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:27.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:27 smithi131 ceph-mon[41654]: pgmap v1644: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:27.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:27 smithi131 ceph-mon[29507]: pgmap v1644: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:29.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:29 smithi157 ceph-mon[33345]: pgmap v1645: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:22:29.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:29 smithi131 ceph-mon[41654]: pgmap v1645: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:22:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:29 smithi131 ceph-mon[29507]: pgmap v1645: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:22:30.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:22:30 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:22:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:31 smithi157 ceph-mon[33345]: pgmap v1646: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:31 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:31 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:22:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:31 smithi131 ceph-mon[41654]: pgmap v1646: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:31 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:31 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:22:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:31 smithi131 ceph-mon[29507]: pgmap v1646: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:31 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:31 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:22:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:33 smithi131 ceph-mon[41654]: pgmap v1647: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:33.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:33 smithi131 ceph-mon[29507]: pgmap v1647: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:33.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:22:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:22:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:22:33.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:33 smithi157 ceph-mon[33345]: pgmap v1647: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:35 smithi157 ceph-mon[33345]: pgmap v1648: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:35.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:35 smithi131 ceph-mon[41654]: pgmap v1648: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:35.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:35 smithi131 ceph-mon[29507]: pgmap v1648: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:37 smithi157 ceph-mon[33345]: pgmap v1649: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:37.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:37 smithi131 ceph-mon[29507]: pgmap v1649: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:37.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:37 smithi131 ceph-mon[41654]: pgmap v1649: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:39.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:39 smithi157 ceph-mon[33345]: pgmap v1650: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:22:39.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:39 smithi131 ceph-mon[29507]: pgmap v1650: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:22:39.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:39 smithi131 ceph-mon[41654]: pgmap v1650: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:22:40.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:22:40 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:22:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:41 smithi157 ceph-mon[33345]: pgmap v1651: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:41 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:41 smithi131 ceph-mon[29507]: pgmap v1651: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:41 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:41 smithi131 ceph-mon[41654]: pgmap v1651: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:41 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:22:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:22:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:22:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:43 smithi131 ceph-mon[29507]: pgmap v1652: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:43 smithi131 ceph-mon[41654]: pgmap v1652: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:43.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:43 smithi157 ceph-mon[33345]: pgmap v1652: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:45.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:45 smithi157 ceph-mon[33345]: pgmap v1653: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:45.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:45 smithi131 ceph-mon[29507]: pgmap v1653: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:45 smithi131 ceph-mon[41654]: pgmap v1653: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:46.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:46 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:22:46.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:46 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:22:46.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:46 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:22:47.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:47 smithi157 ceph-mon[33345]: pgmap v1654: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:47.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:47 smithi131 ceph-mon[41654]: pgmap v1654: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:47.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:47 smithi131 ceph-mon[29507]: pgmap v1654: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:22:49.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:49 smithi157 ceph-mon[33345]: pgmap v1655: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:22:49.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:49 smithi131 ceph-mon[41654]: pgmap v1655: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:22:49.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:49 smithi131 ceph-mon[29507]: pgmap v1655: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:22:49.877 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.878 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.879 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.880 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.881 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.882 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.883 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 1 sec 2024-02-05T08:22:49.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 512 aios 2024-02-05T08:22:49.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559250cfa720 2024-02-05T08:22:49.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510221b0 2024-02-05T08:22:49.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251022b00 2024-02-05T08:22:49.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510230f0 2024-02-05T08:22:49.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251022fe0 2024-02-05T08:22:49.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251023600 2024-02-05T08:22:49.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251023aa0 2024-02-05T08:22:49.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251023fc0 2024-02-05T08:22:49.884 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510244c0 2024-02-05T08:22:49.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251024930 2024-02-05T08:22:49.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251024a40 2024-02-05T08:22:49.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251024ba0 2024-02-05T08:22:49.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510245d0 2024-02-05T08:22:49.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251024760 2024-02-05T08:22:49.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251025190 2024-02-05T08:22:49.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251024cb0 2024-02-05T08:22:49.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251024dc0 2024-02-05T08:22:49.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251024f20 2024-02-05T08:22:49.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510252a0 2024-02-05T08:22:49.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251025760 2024-02-05T08:22:49.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251025870 2024-02-05T08:22:49.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251025e20 2024-02-05T08:22:49.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510253b0 2024-02-05T08:22:49.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251025510 2024-02-05T08:22:49.885 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251025980 2024-02-05T08:22:49.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251025a90 2024-02-05T08:22:49.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251025ba0 2024-02-05T08:22:49.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251025cb0 2024-02-05T08:22:49.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510263d0 2024-02-05T08:22:49.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251025f30 2024-02-05T08:22:49.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251026060 2024-02-05T08:22:49.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251026170 2024-02-05T08:22:49.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510262c0 2024-02-05T08:22:49.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251026980 2024-02-05T08:22:49.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510264e0 2024-02-05T08:22:49.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510265f0 2024-02-05T08:22:49.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251026720 2024-02-05T08:22:49.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251026850 2024-02-05T08:22:49.886 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251026f70 2024-02-05T08:22:49.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251026ad0 2024-02-05T08:22:49.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251026c00 2024-02-05T08:22:49.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251026d10 2024-02-05T08:22:49.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251026e60 2024-02-05T08:22:49.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251027520 2024-02-05T08:22:49.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251027080 2024-02-05T08:22:49.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251027190 2024-02-05T08:22:49.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510272a0 2024-02-05T08:22:49.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510273b0 2024-02-05T08:22:49.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251027af0 2024-02-05T08:22:49.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251027650 2024-02-05T08:22:49.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251027760 2024-02-05T08:22:49.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251027870 2024-02-05T08:22:49.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510279a0 2024-02-05T08:22:49.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510280c0 2024-02-05T08:22:49.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251027c20 2024-02-05T08:22:49.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251027d30 2024-02-05T08:22:49.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251027e60 2024-02-05T08:22:49.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251027f70 2024-02-05T08:22:49.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251028690 2024-02-05T08:22:49.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510281f0 2024-02-05T08:22:49.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251028300 2024-02-05T08:22:49.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251028430 2024-02-05T08:22:49.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251028540 2024-02-05T08:22:49.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251028c60 2024-02-05T08:22:49.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510287c0 2024-02-05T08:22:49.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510288d0 2024-02-05T08:22:49.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251028a20 2024-02-05T08:22:49.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251028b30 2024-02-05T08:22:49.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251029210 2024-02-05T08:22:49.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251028d70 2024-02-05T08:22:49.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251028e80 2024-02-05T08:22:49.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251028fb0 2024-02-05T08:22:49.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510290c0 2024-02-05T08:22:49.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251029810 2024-02-05T08:22:49.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251029990 2024-02-05T08:22:49.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251029aa0 2024-02-05T08:22:49.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251029320 2024-02-05T08:22:49.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251029450 2024-02-05T08:22:49.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251029560 2024-02-05T08:22:49.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251029690 2024-02-05T08:22:49.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102a140 2024-02-05T08:22:49.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251029ca0 2024-02-05T08:22:49.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251029db0 2024-02-05T08:22:49.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251029f00 2024-02-05T08:22:49.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102a010 2024-02-05T08:22:49.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102a6f0 2024-02-05T08:22:49.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102a250 2024-02-05T08:22:49.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102a380 2024-02-05T08:22:49.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102a4b0 2024-02-05T08:22:49.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102a5c0 2024-02-05T08:22:49.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102ace0 2024-02-05T08:22:49.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102a840 2024-02-05T08:22:49.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102a970 2024-02-05T08:22:49.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102aa80 2024-02-05T08:22:49.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102abb0 2024-02-05T08:22:49.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102b290 2024-02-05T08:22:49.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102adf0 2024-02-05T08:22:49.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102af20 2024-02-05T08:22:49.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102b030 2024-02-05T08:22:49.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102b140 2024-02-05T08:22:49.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102b840 2024-02-05T08:22:49.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102b3f0 2024-02-05T08:22:49.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102b500 2024-02-05T08:22:49.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102b630 2024-02-05T08:22:49.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102b990 2024-02-05T08:22:49.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102be00 2024-02-05T08:22:49.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102bf10 2024-02-05T08:22:49.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102c4c0 2024-02-05T08:22:49.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102baa0 2024-02-05T08:22:49.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102bbd0 2024-02-05T08:22:49.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102bce0 2024-02-05T08:22:49.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102c5f0 2024-02-05T08:22:49.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102c020 2024-02-05T08:22:49.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102c170 2024-02-05T08:22:49.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102c280 2024-02-05T08:22:49.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102c390 2024-02-05T08:22:49.893 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102cbc0 2024-02-05T08:22:49.893 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102c720 2024-02-05T08:22:49.893 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102c830 2024-02-05T08:22:49.893 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102c940 2024-02-05T08:22:49.893 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102ca70 2024-02-05T08:22:49.893 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102d190 2024-02-05T08:22:49.893 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102ccf0 2024-02-05T08:22:49.893 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102ce70 2024-02-05T08:22:49.893 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102cff0 2024-02-05T08:22:49.893 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251024110 2024-02-05T08:22:49.893 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251024240 2024-02-05T08:22:49.893 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102db10 2024-02-05T08:22:49.893 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102d670 2024-02-05T08:22:49.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102d7c0 2024-02-05T08:22:49.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102d8d0 2024-02-05T08:22:49.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102da00 2024-02-05T08:22:49.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102e0c0 2024-02-05T08:22:49.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102dc20 2024-02-05T08:22:49.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102dd50 2024-02-05T08:22:49.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102de80 2024-02-05T08:22:49.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102dfb0 2024-02-05T08:22:49.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102e670 2024-02-05T08:22:49.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102e1d0 2024-02-05T08:22:49.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102d2c0 2024-02-05T08:22:49.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102ef80 2024-02-05T08:22:49.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102eb30 2024-02-05T08:22:49.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102d3f0 2024-02-05T08:22:49.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102d500 2024-02-05T08:22:49.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102ec40 2024-02-05T08:22:49.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102ed70 2024-02-05T08:22:49.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102f0d0 2024-02-05T08:22:49.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102f580 2024-02-05T08:22:49.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102fa30 2024-02-05T08:22:49.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102fb40 2024-02-05T08:22:49.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102f1e0 2024-02-05T08:22:49.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102f330 2024-02-05T08:22:49.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102f460 2024-02-05T08:22:49.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102e780 2024-02-05T08:22:49.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102e8b0 2024-02-05T08:22:49.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102ea00 2024-02-05T08:22:49.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251030110 2024-02-05T08:22:49.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102fc70 2024-02-05T08:22:49.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102fd80 2024-02-05T08:22:49.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102feb0 2024-02-05T08:22:49.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102ffe0 2024-02-05T08:22:49.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251030700 2024-02-05T08:22:49.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251030260 2024-02-05T08:22:49.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251030390 2024-02-05T08:22:49.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510304c0 2024-02-05T08:22:49.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510305f0 2024-02-05T08:22:49.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251030cd0 2024-02-05T08:22:49.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251030830 2024-02-05T08:22:49.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251030960 2024-02-05T08:22:49.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251030a70 2024-02-05T08:22:49.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251030ba0 2024-02-05T08:22:49.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510312a0 2024-02-05T08:22:49.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251030e00 2024-02-05T08:22:49.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251030f30 2024-02-05T08:22:49.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251031040 2024-02-05T08:22:49.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251031150 2024-02-05T08:22:49.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251031870 2024-02-05T08:22:49.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510313d0 2024-02-05T08:22:49.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510314e0 2024-02-05T08:22:49.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251031610 2024-02-05T08:22:49.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251031740 2024-02-05T08:22:49.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251031e40 2024-02-05T08:22:49.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510319a0 2024-02-05T08:22:49.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251031af0 2024-02-05T08:22:49.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251031c00 2024-02-05T08:22:49.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251031f70 2024-02-05T08:22:49.897 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251032400 2024-02-05T08:22:49.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251032530 2024-02-05T08:22:49.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251032080 2024-02-05T08:22:49.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510321d0 2024-02-05T08:22:49.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251032b20 2024-02-05T08:22:49.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251032680 2024-02-05T08:22:49.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510327b0 2024-02-05T08:22:49.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510328e0 2024-02-05T08:22:49.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251032a10 2024-02-05T08:22:49.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510330f0 2024-02-05T08:22:49.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251032c50 2024-02-05T08:22:49.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251032d80 2024-02-05T08:22:49.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251032e90 2024-02-05T08:22:49.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251032fe0 2024-02-05T08:22:49.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510336c0 2024-02-05T08:22:49.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251033220 2024-02-05T08:22:49.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251033370 2024-02-05T08:22:49.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251033480 2024-02-05T08:22:49.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510335b0 2024-02-05T08:22:49.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251033c90 2024-02-05T08:22:49.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510337f0 2024-02-05T08:22:49.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251033900 2024-02-05T08:22:49.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251033a50 2024-02-05T08:22:49.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251033de0 2024-02-05T08:22:49.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251034290 2024-02-05T08:22:49.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510343c0 2024-02-05T08:22:49.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251034990 2024-02-05T08:22:49.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251033f30 2024-02-05T08:22:49.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251034040 2024-02-05T08:22:49.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251034170 2024-02-05T08:22:49.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251034ae0 2024-02-05T08:22:49.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510344f0 2024-02-05T08:22:49.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251034640 2024-02-05T08:22:49.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251034750 2024-02-05T08:22:49.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251034bf0 2024-02-05T08:22:49.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251035060 2024-02-05T08:22:49.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251035170 2024-02-05T08:22:49.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251035760 2024-02-05T08:22:49.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251034d00 2024-02-05T08:22:49.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251034e30 2024-02-05T08:22:49.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510352c0 2024-02-05T08:22:49.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510353f0 2024-02-05T08:22:49.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251035500 2024-02-05T08:22:49.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251035610 2024-02-05T08:22:49.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251035d50 2024-02-05T08:22:49.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510358b0 2024-02-05T08:22:49.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510359e0 2024-02-05T08:22:49.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251035af0 2024-02-05T08:22:49.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251035c40 2024-02-05T08:22:49.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251036320 2024-02-05T08:22:49.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251035e80 2024-02-05T08:22:49.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251035fb0 2024-02-05T08:22:49.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510360c0 2024-02-05T08:22:49.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251036210 2024-02-05T08:22:49.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251036910 2024-02-05T08:22:49.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251036470 2024-02-05T08:22:49.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510365a0 2024-02-05T08:22:49.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510366b0 2024-02-05T08:22:49.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251036800 2024-02-05T08:22:49.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251036f00 2024-02-05T08:22:49.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251036a60 2024-02-05T08:22:49.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251036b90 2024-02-05T08:22:49.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251036ca0 2024-02-05T08:22:49.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251036db0 2024-02-05T08:22:49.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510374f0 2024-02-05T08:22:49.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251037050 2024-02-05T08:22:49.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251037160 2024-02-05T08:22:49.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251037290 2024-02-05T08:22:49.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510373e0 2024-02-05T08:22:49.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251037ae0 2024-02-05T08:22:49.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251037640 2024-02-05T08:22:49.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251037770 2024-02-05T08:22:49.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510378a0 2024-02-05T08:22:49.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510379d0 2024-02-05T08:22:49.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251038090 2024-02-05T08:22:49.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251037bf0 2024-02-05T08:22:49.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251037d40 2024-02-05T08:22:49.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251037e50 2024-02-05T08:22:49.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251037f80 2024-02-05T08:22:49.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251038660 2024-02-05T08:22:49.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510381c0 2024-02-05T08:22:49.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510382f0 2024-02-05T08:22:49.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251038400 2024-02-05T08:22:49.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251038550 2024-02-05T08:22:49.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251038c30 2024-02-05T08:22:49.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251038790 2024-02-05T08:22:49.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510388c0 2024-02-05T08:22:49.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251038a10 2024-02-05T08:22:49.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251038d80 2024-02-05T08:22:49.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251038ed0 2024-02-05T08:22:49.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251038b20 2024-02-05T08:22:49.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251039380 2024-02-05T08:22:49.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251039020 2024-02-05T08:22:49.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251039150 2024-02-05T08:22:49.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251039490 2024-02-05T08:22:49.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251039920 2024-02-05T08:22:49.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251039a70 2024-02-05T08:22:49.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102f6b0 2024-02-05T08:22:49.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102f7c0 2024-02-05T08:22:49.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925102f8f0 2024-02-05T08:22:49.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251039bc0 2024-02-05T08:22:49.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103a050 2024-02-05T08:22:49.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103a180 2024-02-05T08:22:49.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103a730 2024-02-05T08:22:49.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251039cf0 2024-02-05T08:22:49.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251039e40 2024-02-05T08:22:49.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103a290 2024-02-05T08:22:49.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103a3c0 2024-02-05T08:22:49.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103a4f0 2024-02-05T08:22:49.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103a880 2024-02-05T08:22:49.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103ad60 2024-02-05T08:22:49.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103aee0 2024-02-05T08:22:49.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103b4b0 2024-02-05T08:22:49.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103a9b0 2024-02-05T08:22:49.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103aae0 2024-02-05T08:22:49.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103abf0 2024-02-05T08:22:49.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103b600 2024-02-05T08:22:49.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103b010 2024-02-05T08:22:49.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103b160 2024-02-05T08:22:49.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103b290 2024-02-05T08:22:49.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103b730 2024-02-05T08:22:49.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103bbc0 2024-02-05T08:22:49.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103bcf0 2024-02-05T08:22:49.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510395c0 2024-02-05T08:22:49.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251039710 2024-02-05T08:22:49.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103c2c0 2024-02-05T08:22:49.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103be20 2024-02-05T08:22:49.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103bf50 2024-02-05T08:22:49.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103c080 2024-02-05T08:22:49.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103c1b0 2024-02-05T08:22:49.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103c890 2024-02-05T08:22:49.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103c3f0 2024-02-05T08:22:49.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103c500 2024-02-05T08:22:49.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103c630 2024-02-05T08:22:49.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103c780 2024-02-05T08:22:49.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103ce40 2024-02-05T08:22:49.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103c9a0 2024-02-05T08:22:49.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103caf0 2024-02-05T08:22:49.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103cc00 2024-02-05T08:22:49.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103cf90 2024-02-05T08:22:49.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103d440 2024-02-05T08:22:49.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103d590 2024-02-05T08:22:49.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103d0c0 2024-02-05T08:22:49.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103d210 2024-02-05T08:22:49.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103db60 2024-02-05T08:22:49.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103d6c0 2024-02-05T08:22:49.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103d810 2024-02-05T08:22:49.908 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103d920 2024-02-05T08:22:49.908 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103da30 2024-02-05T08:22:49.908 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103e150 2024-02-05T08:22:49.908 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103dcb0 2024-02-05T08:22:49.908 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103dde0 2024-02-05T08:22:49.908 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103df10 2024-02-05T08:22:49.908 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103e040 2024-02-05T08:22:49.908 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103e720 2024-02-05T08:22:49.908 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103e280 2024-02-05T08:22:49.908 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103e3d0 2024-02-05T08:22:49.908 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103e500 2024-02-05T08:22:49.908 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103e850 2024-02-05T08:22:49.908 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103ece0 2024-02-05T08:22:49.908 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103ee10 2024-02-05T08:22:49.909 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103f3e0 2024-02-05T08:22:49.909 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103e980 2024-02-05T08:22:49.909 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103eab0 2024-02-05T08:22:49.909 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103ef40 2024-02-05T08:22:49.909 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103f070 2024-02-05T08:22:49.909 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103f1a0 2024-02-05T08:22:49.909 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103f2d0 2024-02-05T08:22:49.909 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103f9b0 2024-02-05T08:22:49.909 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103f510 2024-02-05T08:22:49.909 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103f660 2024-02-05T08:22:49.909 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103f790 2024-02-05T08:22:49.909 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103fae0 2024-02-05T08:22:49.909 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103ff70 2024-02-05T08:22:49.909 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510400a0 2024-02-05T08:22:49.909 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103e610 2024-02-05T08:22:49.910 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510401d0 2024-02-05T08:22:49.910 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103b860 2024-02-05T08:22:49.910 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103b990 2024-02-05T08:22:49.910 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103baa0 2024-02-05T08:22:49.910 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510407a0 2024-02-05T08:22:49.910 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251040300 2024-02-05T08:22:49.910 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251040430 2024-02-05T08:22:49.910 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251040540 2024-02-05T08:22:49.910 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251040690 2024-02-05T08:22:49.910 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251040d50 2024-02-05T08:22:49.910 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510408b0 2024-02-05T08:22:49.910 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251040a00 2024-02-05T08:22:49.910 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251040b30 2024-02-05T08:22:49.910 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251040ea0 2024-02-05T08:22:49.910 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251041330 2024-02-05T08:22:49.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251041460 2024-02-05T08:22:49.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251041a30 2024-02-05T08:22:49.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251040fd0 2024-02-05T08:22:49.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251041100 2024-02-05T08:22:49.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251041590 2024-02-05T08:22:49.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510416c0 2024-02-05T08:22:49.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510417f0 2024-02-05T08:22:49.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251041920 2024-02-05T08:22:49.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251041fe0 2024-02-05T08:22:49.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251041b40 2024-02-05T08:22:49.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251041c70 2024-02-05T08:22:49.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251041da0 2024-02-05T08:22:49.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251041ed0 2024-02-05T08:22:49.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510425b0 2024-02-05T08:22:49.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251042110 2024-02-05T08:22:49.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251042240 2024-02-05T08:22:49.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251042350 2024-02-05T08:22:49.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251042480 2024-02-05T08:22:49.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251042b80 2024-02-05T08:22:49.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510426e0 2024-02-05T08:22:49.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251042810 2024-02-05T08:22:49.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251042960 2024-02-05T08:22:49.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251042cb0 2024-02-05T08:22:49.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251043140 2024-02-05T08:22:49.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251043270 2024-02-05T08:22:49.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251042a70 2024-02-05T08:22:49.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510433e0 2024-02-05T08:22:49.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251043530 2024-02-05T08:22:49.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251043b20 2024-02-05T08:22:49.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103fc10 2024-02-05T08:22:49.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925103fd40 2024-02-05T08:22:49.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251043680 2024-02-05T08:22:49.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510437b0 2024-02-05T08:22:49.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510438e0 2024-02-05T08:22:49.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251043a10 2024-02-05T08:22:49.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510440f0 2024-02-05T08:22:49.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251043c50 2024-02-05T08:22:49.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251043d80 2024-02-05T08:22:49.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251043e90 2024-02-05T08:22:49.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251044220 2024-02-05T08:22:49.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510446b0 2024-02-05T08:22:49.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251044800 2024-02-05T08:22:49.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251044dd0 2024-02-05T08:22:49.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251044350 2024-02-05T08:22:49.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251044480 2024-02-05T08:22:49.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251044930 2024-02-05T08:22:49.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251044a60 2024-02-05T08:22:49.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251044b90 2024-02-05T08:22:49.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251044cc0 2024-02-05T08:22:49.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510453a0 2024-02-05T08:22:49.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251044f00 2024-02-05T08:22:49.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251045050 2024-02-05T08:22:49.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251045180 2024-02-05T08:22:49.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510454d0 2024-02-05T08:22:49.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251045960 2024-02-05T08:22:49.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251045a90 2024-02-05T08:22:49.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251044590 2024-02-05T08:22:49.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251045f20 2024-02-05T08:22:49.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251045bc0 2024-02-05T08:22:49.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251045cf0 2024-02-05T08:22:49.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251045e00 2024-02-05T08:22:49.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510455e0 2024-02-05T08:22:49.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251045750 2024-02-05T08:22:49.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251043fc0 2024-02-05T08:22:49.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251046030 2024-02-05T08:22:49.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510464e0 2024-02-05T08:22:49.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251046610 2024-02-05T08:22:49.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251046be0 2024-02-05T08:22:49.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251046180 2024-02-05T08:22:49.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510462b0 2024-02-05T08:22:50.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:50 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:22:50.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:22:50 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:22:50.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:50 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:22:50.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:50 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:22:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:51 smithi157 ceph-mon[33345]: pgmap v1656: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:51 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:51 smithi157 ceph-mon[33345]: osdmap e756: 8 total, 8 up, 8 in 2024-02-05T08:22:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:51 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:22:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:51 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:22:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:51 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:22:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:51 smithi131 ceph-mon[29507]: pgmap v1656: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:51 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:51 smithi131 ceph-mon[29507]: osdmap e756: 8 total, 8 up, 8 in 2024-02-05T08:22:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:51 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:22:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:51 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:22:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:51 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:22:51.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:51 smithi131 ceph-mon[41654]: pgmap v1656: 228 pgs: 228 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:22:51.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:51 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:22:51.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:51 smithi131 ceph-mon[41654]: osdmap e756: 8 total, 8 up, 8 in 2024-02-05T08:22:51.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:51 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:22:51.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:51 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:22:51.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:51 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:22:52.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:52 smithi157 ceph-mon[33345]: osdmap e757: 8 total, 8 up, 8 in 2024-02-05T08:22:52.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:52 smithi131 ceph-mon[41654]: osdmap e757: 8 total, 8 up, 8 in 2024-02-05T08:22:52.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:52 smithi131 ceph-mon[29507]: osdmap e757: 8 total, 8 up, 8 in 2024-02-05T08:22:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:22:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:22:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:22:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:53 smithi131 ceph-mon[41654]: pgmap v1659: 164 pgs: 164 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:22:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:53 smithi131 ceph-mon[41654]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:22:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:53 smithi131 ceph-mon[41654]: osdmap e758: 8 total, 8 up, 8 in 2024-02-05T08:22:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:22:53.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:53 smithi131 ceph-mon[29507]: pgmap v1659: 164 pgs: 164 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:22:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:53 smithi131 ceph-mon[29507]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:22:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:53 smithi131 ceph-mon[29507]: osdmap e758: 8 total, 8 up, 8 in 2024-02-05T08:22:53.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:22:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:53 smithi157 ceph-mon[33345]: pgmap v1659: 164 pgs: 164 active+clean; 582 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:22:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:53 smithi157 ceph-mon[33345]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:22:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:53 smithi157 ceph-mon[33345]: osdmap e758: 8 total, 8 up, 8 in 2024-02-05T08:22:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:22:54.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:22:54.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:54 smithi157 ceph-mon[33345]: osdmap e759: 8 total, 8 up, 8 in 2024-02-05T08:22:54.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-05T08:22:54.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:54 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:22:54.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:22:54.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:54 smithi131 ceph-mon[41654]: osdmap e759: 8 total, 8 up, 8 in 2024-02-05T08:22:54.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-05T08:22:54.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:54 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:22:54.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:22:54.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:54 smithi131 ceph-mon[29507]: osdmap e759: 8 total, 8 up, 8 in 2024-02-05T08:22:54.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-05T08:22:54.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:54 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:22:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:55 smithi157 ceph-mon[33345]: pgmap v1662: 196 pgs: 196 active+clean; 583 KiB data, 861 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:22:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-05T08:22:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:55 smithi157 ceph-mon[33345]: osdmap e760: 8 total, 8 up, 8 in 2024-02-05T08:22:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "app2"}]: dispatch 2024-02-05T08:22:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:55 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:22:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:55 smithi131 ceph-mon[41654]: pgmap v1662: 196 pgs: 196 active+clean; 583 KiB data, 861 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:22:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-05T08:22:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:55 smithi131 ceph-mon[41654]: osdmap e760: 8 total, 8 up, 8 in 2024-02-05T08:22:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "app2"}]: dispatch 2024-02-05T08:22:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:55 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:22:55.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:55 smithi131 ceph-mon[29507]: pgmap v1662: 196 pgs: 196 active+clean; 583 KiB data, 861 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:22:55.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-05T08:22:55.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:55 smithi131 ceph-mon[29507]: osdmap e760: 8 total, 8 up, 8 in 2024-02-05T08:22:55.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "app2"}]: dispatch 2024-02-05T08:22:55.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:55 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:22:56.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-05T08:22:56.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:56 smithi157 ceph-mon[33345]: osdmap e761: 8 total, 8 up, 8 in 2024-02-05T08:22:56.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-82238-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-05T08:22:56.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:56 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T08:22:56.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-05T08:22:56.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:56 smithi131 ceph-mon[41654]: osdmap e761: 8 total, 8 up, 8 in 2024-02-05T08:22:56.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-82238-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-05T08:22:56.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:56 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T08:22:56.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-82238-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-05T08:22:56.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:56 smithi131 ceph-mon[29507]: osdmap e761: 8 total, 8 up, 8 in 2024-02-05T08:22:56.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-82238-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-05T08:22:56.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:56 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T08:22:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:57 smithi157 ceph-mon[33345]: pgmap v1665: 196 pgs: 196 active+clean; 583 KiB data, 861 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:22:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:57 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-05T08:22:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:57 smithi157 ceph-mon[33345]: osdmap e762: 8 total, 8 up, 8 in 2024-02-05T08:22:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:57 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T08:22:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:57 smithi131 ceph-mon[41654]: pgmap v1665: 196 pgs: 196 active+clean; 583 KiB data, 861 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:22:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:57 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-05T08:22:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:57 smithi131 ceph-mon[41654]: osdmap e762: 8 total, 8 up, 8 in 2024-02-05T08:22:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:57 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T08:22:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:57 smithi131 ceph-mon[29507]: pgmap v1665: 196 pgs: 196 active+clean; 583 KiB data, 861 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:22:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:57 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-05T08:22:57.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:57 smithi131 ceph-mon[29507]: osdmap e762: 8 total, 8 up, 8 in 2024-02-05T08:22:57.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:57 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T08:22:58.182 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251046740 2024-02-05T08:22:58.182 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251046870 2024-02-05T08:22:58.182 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510469a0 2024-02-05T08:22:58.182 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251046ad0 2024-02-05T08:22:58.182 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510471d0 2024-02-05T08:22:58.182 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251046d30 2024-02-05T08:22:58.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251046e80 2024-02-05T08:22:58.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251046fb0 2024-02-05T08:22:58.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251047300 2024-02-05T08:22:58.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251047790 2024-02-05T08:22:58.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510478c0 2024-02-05T08:22:58.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251047e90 2024-02-05T08:22:58.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251047430 2024-02-05T08:22:58.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251047560 2024-02-05T08:22:58.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510479f0 2024-02-05T08:22:58.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251047b20 2024-02-05T08:22:58.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251047670 2024-02-05T08:22:58.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251047c50 2024-02-05T08:22:58.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251048460 2024-02-05T08:22:58.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251047fc0 2024-02-05T08:22:58.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510480d0 2024-02-05T08:22:58.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251048200 2024-02-05T08:22:58.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251048350 2024-02-05T08:22:58.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251048a30 2024-02-05T08:22:58.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251048590 2024-02-05T08:22:58.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510486c0 2024-02-05T08:22:58.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510487d0 2024-02-05T08:22:58.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251048920 2024-02-05T08:22:58.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251049020 2024-02-05T08:22:58.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251048b80 2024-02-05T08:22:58.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251048cb0 2024-02-05T08:22:58.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251048e00 2024-02-05T08:22:58.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251049170 2024-02-05T08:22:58.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251049600 2024-02-05T08:22:58.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251049730 2024-02-05T08:22:58.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251049280 2024-02-05T08:22:58.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x5592510493d0 2024-02-05T08:22:58.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251049d00 2024-02-05T08:22:58.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251049860 2024-02-05T08:22:58.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251049990 2024-02-05T08:22:58.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251049ac0 2024-02-05T08:22:58.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251049bf0 2024-02-05T08:22:58.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559250e74600 2024-02-05T08:22:58.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559250e74c30 2024-02-05T08:22:58.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559250e74790 2024-02-05T08:22:58.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559250e748a0 2024-02-05T08:22:58.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559250e749f0 2024-02-05T08:22:58.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559250e74b20 2024-02-05T08:22:58.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559250e75200 2024-02-05T08:22:58.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559250e75330 2024-02-05T08:22:58.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559250e75460 2024-02-05T08:22:58.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559250e755b0 2024-02-05T08:22:58.186 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251049e30 2024-02-05T08:22:58.186 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559250e74d60 2024-02-05T08:22:58.186 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559250e74eb0 2024-02-05T08:22:58.186 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559250e75000 2024-02-05T08:22:58.186 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x559251049f80 2024-02-05T08:22:58.186 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925104a410 2024-02-05T08:22:58.186 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925104a540 2024-02-05T08:22:58.186 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925104a090 2024-02-05T08:22:58.186 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925104a1e0 2024-02-05T08:22:58.186 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925104ab10 2024-02-05T08:22:58.186 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925104a670 2024-02-05T08:22:58.186 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925104a7a0 2024-02-05T08:22:58.186 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55925104a8d0 2024-02-05T08:22:58.186 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: done. 2024-02-05T08:22:58.186 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (1810274 ms) 2024-02-05T08:22:58.187 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (1810274 ms total) 2024-02-05T08:22:58.187 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: 2024-02-05T08:22:58.187 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-02-05T08:22:58.187 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-02-05T08:22:58.187 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-02-05T08:22:58.187 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-02-05T08:22:58.187 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMisc.Exec (17 ms) 2024-02-05T08:22:58.187 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-02-05T08:22:58.187 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (4 ms) 2024-02-05T08:22:58.187 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-02-05T08:22:58.187 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (3 ms) 2024-02-05T08:22:58.187 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-02-05T08:22:58.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-05T08:22:58.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:58 smithi157 ceph-mon[33345]: osdmap e763: 8 total, 8 up, 8 in 2024-02-05T08:22:58.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T08:22:58.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key1"}]': finished 2024-02-05T08:22:58.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-05T08:22:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:58 smithi131 ceph-mon[41654]: osdmap e763: 8 total, 8 up, 8 in 2024-02-05T08:22:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T08:22:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key1"}]': finished 2024-02-05T08:22:58.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-05T08:22:58.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:58 smithi131 ceph-mon[29507]: osdmap e763: 8 total, 8 up, 8 in 2024-02-05T08:22:58.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T08:22:58.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/4221419846' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi131-82238-1","app":"app1","key":"key1"}]': finished 2024-02-05T08:22:59.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:59 smithi157 ceph-mon[33345]: pgmap v1668: 196 pgs: 196 active+clean; 583 KiB data, 861 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:22:59.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:22:59 smithi157 ceph-mon[33345]: osdmap e764: 8 total, 8 up, 8 in 2024-02-05T08:22:59.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:59 smithi131 ceph-mon[29507]: pgmap v1668: 196 pgs: 196 active+clean; 583 KiB data, 861 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:22:59.516 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:22:59 smithi131 ceph-mon[29507]: osdmap e764: 8 total, 8 up, 8 in 2024-02-05T08:22:59.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:59 smithi131 ceph-mon[41654]: pgmap v1668: 196 pgs: 196 active+clean; 583 KiB data, 861 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:22:59.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:22:59 smithi131 ceph-mon[41654]: osdmap e764: 8 total, 8 up, 8 in 2024-02-05T08:23:00.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:23:00 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:23:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:01 smithi157 ceph-mon[33345]: pgmap v1670: 196 pgs: 196 active+clean; 583 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:23:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:01 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:01 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:23:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:01 smithi131 ceph-mon[41654]: pgmap v1670: 196 pgs: 196 active+clean; 583 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:23:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:01 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:01 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:23:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:01 smithi131 ceph-mon[29507]: pgmap v1670: 196 pgs: 196 active+clean; 583 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:23:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:01 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:01 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:23:03.195 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:03 smithi131 ceph-mon[29507]: pgmap v1671: 196 pgs: 196 active+clean; 583 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:03.195 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:23:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:23:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:23:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:03 smithi157 ceph-mon[33345]: pgmap v1671: 196 pgs: 196 active+clean; 583 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:03.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:03 smithi131 ceph-mon[41654]: pgmap v1671: 196 pgs: 196 active+clean; 583 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:05 smithi157 ceph-mon[33345]: pgmap v1672: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:23:05.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:05 smithi131 ceph-mon[41654]: pgmap v1672: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:23:05.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:05 smithi131 ceph-mon[29507]: pgmap v1672: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:23:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:07 smithi157 ceph-mon[33345]: pgmap v1673: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:23:07.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:07 smithi131 ceph-mon[41654]: pgmap v1673: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:23:07.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:07 smithi131 ceph-mon[29507]: pgmap v1673: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:23:09.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:09 smithi157 ceph-mon[33345]: pgmap v1674: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:23:09.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:09 smithi131 ceph-mon[41654]: pgmap v1674: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:23:09.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:09 smithi131 ceph-mon[29507]: pgmap v1674: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:23:10.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:23:10 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:23:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:11 smithi157 ceph-mon[33345]: pgmap v1675: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:23:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:11 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:11 smithi131 ceph-mon[41654]: pgmap v1675: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:23:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:11 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:11 smithi131 ceph-mon[29507]: pgmap v1675: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:23:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:11 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:13.207 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:23:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:23:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:23:13.207 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:13 smithi131 ceph-mon[29507]: pgmap v1676: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:13.208 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:13 smithi131 ceph-mon[41654]: pgmap v1676: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:13.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:13 smithi157 ceph-mon[33345]: pgmap v1676: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:15 smithi157 ceph-mon[33345]: pgmap v1677: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:15.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:15 smithi131 ceph-mon[29507]: pgmap v1677: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:15.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:15 smithi131 ceph-mon[41654]: pgmap v1677: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:16.234 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMisc.Applications (5010 ms) 2024-02-05T08:23:16.235 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-02-05T08:23:16.235 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-02-05T08:23:16.235 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-02-05T08:23:16.235 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-02-05T08:23:16.235 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-02-05T08:23:16.235 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (17670 ms) 2024-02-05T08:23:16.235 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] 8 tests from LibRadosMisc (22706 ms total) 2024-02-05T08:23:16.235 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: 2024-02-05T08:23:16.235 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] Global test environment tear-down 2024-02-05T08:23:16.235 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (1837443 ms total) 2024-02-05T08:23:16.235 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ PASSED ] 12 tests. 2024-02-05T08:23:16.240 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.240 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82372 2024-02-05T08:23:16.240 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82372 2024-02-05T08:23:16.240 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.240 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82876 2024-02-05T08:23:16.240 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82876 2024-02-05T08:23:16.240 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.240 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82911 2024-02-05T08:23:16.240 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82911 2024-02-05T08:23:16.240 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.240 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82116 2024-02-05T08:23:16.240 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82116 2024-02-05T08:23:16.241 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.241 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82768 2024-02-05T08:23:16.241 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82768 2024-02-05T08:23:16.241 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.241 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82635 2024-02-05T08:23:16.241 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82635 2024-02-05T08:23:16.241 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.241 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=81983 2024-02-05T08:23:16.241 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 81983 2024-02-05T08:23:16.241 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.241 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82048 2024-02-05T08:23:16.241 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82048 2024-02-05T08:23:16.241 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.241 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82645 2024-02-05T08:23:16.241 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82645 2024-02-05T08:23:16.241 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=81974 2024-02-05T08:23:16.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 81974 2024-02-05T08:23:16.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=81994 2024-02-05T08:23:16.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 81994 2024-02-05T08:23:16.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=83114 2024-02-05T08:23:16.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 83114 2024-02-05T08:23:16.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82262 2024-02-05T08:23:16.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82262 2024-02-05T08:23:16.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=83178 2024-02-05T08:23:16.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 83178 2024-02-05T08:23:16.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82357 2024-02-05T08:23:16.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82357 2024-02-05T08:23:16.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82837 2024-02-05T08:23:16.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82837 2024-02-05T08:23:16.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=83235 2024-02-05T08:23:16.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 83235 2024-02-05T08:23:16.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82277 2024-02-05T08:23:16.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82277 2024-02-05T08:23:16.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82698 2024-02-05T08:23:16.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82698 2024-02-05T08:23:16.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82999 2024-02-05T08:23:16.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82999 2024-02-05T08:23:16.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82798 2024-02-05T08:23:16.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82798 2024-02-05T08:23:16.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=83151 2024-02-05T08:23:16.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 83151 2024-02-05T08:23:16.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82618 2024-02-05T08:23:16.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82618 2024-02-05T08:23:16.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82183 2024-02-05T08:23:16.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82183 2024-02-05T08:23:16.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=83195 2024-02-05T08:23:16.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 83195 2024-02-05T08:23:16.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82500 2024-02-05T08:23:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82500 2024-02-05T08:23:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=81967 2024-02-05T08:23:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 81967 2024-02-05T08:23:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82026 2024-02-05T08:23:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82026 2024-02-05T08:23:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82397 2024-02-05T08:23:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82397 2024-02-05T08:23:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82066 2024-02-05T08:23:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82066 2024-02-05T08:23:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.246 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82530 2024-02-05T08:23:16.246 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82530 2024-02-05T08:23:16.246 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.246 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82568 2024-02-05T08:23:16.246 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82568 2024-02-05T08:23:16.246 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.246 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=82966 2024-02-05T08:23:16.246 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 82966 2024-02-05T08:23:16.246 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:23:16.246 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=83267 2024-02-05T08:23:16.246 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 83267 2024-02-05T08:23:16.246 INFO:tasks.workunit.client.0.smithi131.stderr:+ exit 0 2024-02-05T08:23:16.246 INFO:tasks.workunit.client.0.smithi131.stderr:+ cleanup 2024-02-05T08:23:16.246 INFO:tasks.workunit.client.0.smithi131.stderr:+ pkill -P 81962 2024-02-05T08:23:16.259 INFO:tasks.workunit.client.0.smithi131.stderr:+ true 2024-02-05T08:23:16.260 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-05T08:23:16.260 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-05T08:23:16.290 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-02-05T08:23:16.291 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-02-05T08:23:16.292 DEBUG:teuthology.orchestra.run.smithi131:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=164d3be4a527be775f4d3c343ea591122f1b8921 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-02-05T08:23:16.350 INFO:tasks.workunit.client.0.smithi131.stderr:++ uuidgen 2024-02-05T08:23:16.354 INFO:tasks.workunit.client.0.smithi131.stderr:+ p=b9122de8-c2e1-4670-8261-fe325aa3ccfb 2024-02-05T08:23:16.355 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool create b9122de8-c2e1-4670-8261-fe325aa3ccfb 12 2024-02-05T08:23:16.438 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.436+0000 7fa8772c9640 1 -- 172.21.15.131:0/2089957345 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa870101e20 msgr2=0x7fa870105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:16.438 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.436+0000 7fa8772c9640 1 --2- 172.21.15.131:0/2089957345 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa870101e20 0x7fa870105e60 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009900 tx=0x7fa86c02bc30 comp rx=0 tx=0).stop 2024-02-05T08:23:16.438 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.436+0000 7fa8772c9640 1 -- 172.21.15.131:0/2089957345 shutdown_connections 2024-02-05T08:23:16.438 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.436+0000 7fa8772c9640 1 --2- 172.21.15.131:0/2089957345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa870106540 0x7fa87010e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:16.439 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.436+0000 7fa8772c9640 1 --2- 172.21.15.131:0/2089957345 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa870101e20 0x7fa870105e60 unknown :-1 s=CLOSED pgs=3058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:16.439 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.436+0000 7fa8772c9640 1 --2- 172.21.15.131:0/2089957345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa870101470 0x7fa870101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:16.439 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.436+0000 7fa8772c9640 1 -- 172.21.15.131:0/2089957345 >> 172.21.15.131:0/2089957345 conn(0x7fa8700fcd30 msgr2=0x7fa8700ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:23:16.439 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.436+0000 7fa8772c9640 1 -- 172.21.15.131:0/2089957345 shutdown_connections 2024-02-05T08:23:16.439 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.436+0000 7fa8772c9640 1 -- 172.21.15.131:0/2089957345 wait complete. 2024-02-05T08:23:16.439 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.436+0000 7fa8772c9640 1 Processor -- start 2024-02-05T08:23:16.439 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.436+0000 7fa8772c9640 1 -- start start 2024-02-05T08:23:16.439 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.436+0000 7fa8772c9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa870101470 0x7fa870197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:16.439 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa8772c9640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa870106540 0x7fa870197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:16.439 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa8772c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa87019b8f0 0x7fa87019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:16.439 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa8772c9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87019c2e0 con 0x7fa870101470 2024-02-05T08:23:16.439 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa8772c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87019c450 con 0x7fa87019b8f0 2024-02-05T08:23:16.439 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa8772c9640 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87019c5c0 con 0x7fa870106540 2024-02-05T08:23:16.440 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa875065640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa870101470 0x7fa870197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:16.440 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa874864640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa870106540 0x7fa870197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:16.440 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa875065640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa870101470 0x7fa870197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43802/0 (socket says 172.21.15.131:43802) 2024-02-05T08:23:16.440 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa875065640 1 -- 172.21.15.131:0/1093003226 learned_addr learned my addr 172.21.15.131:0/1093003226 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T08:23:16.440 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa875866640 1 --2- 172.21.15.131:0/1093003226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa87019b8f0 0x7fa87019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:16.440 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa875065640 1 -- 172.21.15.131:0/1093003226 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa870106540 msgr2=0x7fa870197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:16.440 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa875065640 1 --2- 172.21.15.131:0/1093003226 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa870106540 0x7fa870197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:16.440 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa875065640 1 -- 172.21.15.131:0/1093003226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa87019b8f0 msgr2=0x7fa87019bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:16.440 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa875065640 1 --2- 172.21.15.131:0/1093003226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa87019b8f0 0x7fa87019bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:16.440 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa875065640 1 -- 172.21.15.131:0/1093003226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa868009840 con 0x7fa870101470 2024-02-05T08:23:16.440 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa874864640 1 --2- 172.21.15.131:0/1093003226 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa870106540 0x7fa870197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:23:16.440 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa875866640 1 --2- 172.21.15.131:0/1093003226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa87019b8f0 0x7fa87019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T08:23:16.440 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa875065640 1 --2- 172.21.15.131:0/1093003226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa870101470 0x7fa870197030 secure :-1 s=READY pgs=3072 cs=0 l=1 rev1=1 crypto rx=0x7fa86400b4a0 tx=0x7fa86400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:23:16.440 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa8627fc640 1 -- 172.21.15.131:0/1093003226 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864010610 con 0x7fa870101470 2024-02-05T08:23:16.440 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa8772c9640 1 -- 172.21.15.131:0/1093003226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c009560 con 0x7fa870101470 2024-02-05T08:23:16.441 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa8627fc640 1 -- 172.21.15.131:0/1093003226 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fa864010c20 con 0x7fa870101470 2024-02-05T08:23:16.441 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa8627fc640 1 -- 172.21.15.131:0/1093003226 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864017cd0 con 0x7fa870101470 2024-02-05T08:23:16.441 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.437+0000 7fa8772c9640 1 -- 172.21.15.131:0/1093003226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8701a3e90 con 0x7fa870101470 2024-02-05T08:23:16.441 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.438+0000 7fa8772c9640 1 -- 172.21.15.131:0/1093003226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa870068560 con 0x7fa870101470 2024-02-05T08:23:16.441 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.439+0000 7fa8627fc640 1 -- 172.21.15.131:0/1093003226 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fa864010770 con 0x7fa870101470 2024-02-05T08:23:16.442 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.439+0000 7fa8627fc640 1 --2- 172.21.15.131:0/1093003226 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa85c06d9f0 0x7fa85c06feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:16.442 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.439+0000 7fa874864640 1 --2- 172.21.15.131:0/1093003226 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa85c06d9f0 0x7fa85c06feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:16.442 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.439+0000 7fa8627fc640 1 -- 172.21.15.131:0/1093003226 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(765..765 src has 254..765) v4 ==== 7426+0+0 (secure 0 0 0) 0x7fa86408ced0 con 0x7fa870101470 2024-02-05T08:23:16.442 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.440+0000 7fa874864640 1 --2- 172.21.15.131:0/1093003226 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa85c06d9f0 0x7fa85c06feb0 secure :-1 s=READY pgs=4062 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009230 tx=0x7fa86c005fd0 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:23:16.444 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.444+0000 7fa8627fc640 1 -- 172.21.15.131:0/1093003226 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa864059f10 con 0x7fa870101470 2024-02-05T08:23:16.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:23:16.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:23:16.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:23:16.618 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:16.618+0000 7fa8772c9640 1 -- 172.21.15.131:0/1093003226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pg_num": 12} v 0) v1 -- 0x7fa870101e20 con 0x7fa870101470 2024-02-05T08:23:17.218 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.218+0000 7fa8627fc640 1 -- 172.21.15.131:0/1093003226 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pg_num": 12}]=0 pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' created v766) v1 ==== 176+0+0 (secure 0 0 0) 0x7fa864059aa0 con 0x7fa870101470 2024-02-05T08:23:17.325 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.325+0000 7fa8772c9640 1 -- 172.21.15.131:0/1093003226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pg_num": 12} v 0) v1 -- 0x7fa870101fb0 con 0x7fa870101470 2024-02-05T08:23:17.326 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.326+0000 7fa8627fc640 1 -- 172.21.15.131:0/1093003226 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pg_num": 12}]=0 pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' already exists v766) v1 ==== 183+0+0 (secure 0 0 0) 0x7fa8640140c0 con 0x7fa870101470 2024-02-05T08:23:17.326 INFO:tasks.workunit.client.0.smithi131.stderr:pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' already exists 2024-02-05T08:23:17.328 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.328+0000 7fa8772c9640 1 -- 172.21.15.131:0/1093003226 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa85c06d9f0 msgr2=0x7fa85c06feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:17.328 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.328+0000 7fa8772c9640 1 --2- 172.21.15.131:0/1093003226 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa85c06d9f0 0x7fa85c06feb0 secure :-1 s=READY pgs=4062 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009230 tx=0x7fa86c005fd0 comp rx=0 tx=0).stop 2024-02-05T08:23:17.328 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.328+0000 7fa8772c9640 1 -- 172.21.15.131:0/1093003226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa870101470 msgr2=0x7fa870197030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:17.328 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.328+0000 7fa8772c9640 1 --2- 172.21.15.131:0/1093003226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa870101470 0x7fa870197030 secure :-1 s=READY pgs=3072 cs=0 l=1 rev1=1 crypto rx=0x7fa86400b4a0 tx=0x7fa86400b960 comp rx=0 tx=0).stop 2024-02-05T08:23:17.328 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.328+0000 7fa8772c9640 1 -- 172.21.15.131:0/1093003226 shutdown_connections 2024-02-05T08:23:17.328 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.328+0000 7fa8772c9640 1 --2- 172.21.15.131:0/1093003226 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa85c06d9f0 0x7fa85c06feb0 unknown :-1 s=CLOSED pgs=4062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:17.328 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.328+0000 7fa8772c9640 1 --2- 172.21.15.131:0/1093003226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa87019b8f0 0x7fa87019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:17.328 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.328+0000 7fa8772c9640 1 --2- 172.21.15.131:0/1093003226 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa870106540 0x7fa870197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:17.328 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.328+0000 7fa8772c9640 1 --2- 172.21.15.131:0/1093003226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa870101470 0x7fa870197030 unknown :-1 s=CLOSED pgs=3072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:17.328 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.328+0000 7fa8772c9640 1 -- 172.21.15.131:0/1093003226 >> 172.21.15.131:0/1093003226 conn(0x7fa8700fcd30 msgr2=0x7fa8700fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:23:17.329 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.328+0000 7fa8772c9640 1 -- 172.21.15.131:0/1093003226 shutdown_connections 2024-02-05T08:23:17.329 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.328+0000 7fa8772c9640 1 -- 172.21.15.131:0/1093003226 wait complete. 2024-02-05T08:23:17.337 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool set-quota b9122de8-c2e1-4670-8261-fe325aa3ccfb max_objects 10 2024-02-05T08:23:17.421 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.419+0000 7fa794d7a640 1 -- 172.21.15.131:0/3380188513 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa790101e20 msgr2=0x7fa790105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:17.421 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.419+0000 7fa794d7a640 1 --2- 172.21.15.131:0/3380188513 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa790101e20 0x7fa790105e60 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 crypto rx=0x7fa778009900 tx=0x7fa77802bc30 comp rx=0 tx=0).stop 2024-02-05T08:23:17.421 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.419+0000 7fa794d7a640 1 -- 172.21.15.131:0/3380188513 shutdown_connections 2024-02-05T08:23:17.421 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.419+0000 7fa794d7a640 1 --2- 172.21.15.131:0/3380188513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa790106540 0x7fa79010e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:17.422 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.419+0000 7fa794d7a640 1 --2- 172.21.15.131:0/3380188513 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa790101e20 0x7fa790105e60 unknown :-1 s=CLOSED pgs=3059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:17.422 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.419+0000 7fa794d7a640 1 --2- 172.21.15.131:0/3380188513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa790101470 0x7fa790101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:17.422 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.419+0000 7fa794d7a640 1 -- 172.21.15.131:0/3380188513 >> 172.21.15.131:0/3380188513 conn(0x7fa7900fcd30 msgr2=0x7fa7900ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:23:17.422 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.419+0000 7fa794d7a640 1 -- 172.21.15.131:0/3380188513 shutdown_connections 2024-02-05T08:23:17.422 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.419+0000 7fa794d7a640 1 -- 172.21.15.131:0/3380188513 wait complete. 2024-02-05T08:23:17.422 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.419+0000 7fa794d7a640 1 Processor -- start 2024-02-05T08:23:17.422 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.419+0000 7fa794d7a640 1 -- start start 2024-02-05T08:23:17.422 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa794d7a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa790101470 0x7fa790196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:17.422 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa794d7a640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa790106540 0x7fa790197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:17.422 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa794d7a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79019b890 0x7fa79019bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:17.422 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa794d7a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79019c280 con 0x7fa79019b890 2024-02-05T08:23:17.422 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa794d7a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79019c3f0 con 0x7fa790101470 2024-02-05T08:23:17.422 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa794d7a640 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79019c560 con 0x7fa790106540 2024-02-05T08:23:17.422 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa787fff640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa790106540 0x7fa790197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:17.423 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa78ed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79019b890 0x7fa79019bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:17.423 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa787fff640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa790106540 0x7fa790197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:49480/0 (socket says 172.21.15.131:49480) 2024-02-05T08:23:17.423 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa787fff640 1 -- 172.21.15.131:0/908157902 learned_addr learned my addr 172.21.15.131:0/908157902 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T08:23:17.423 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa78e59c640 1 --2- 172.21.15.131:0/908157902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa790101470 0x7fa790196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:17.423 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa78ed9d640 1 -- 172.21.15.131:0/908157902 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa790106540 msgr2=0x7fa790197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:17.423 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa78ed9d640 1 --2- 172.21.15.131:0/908157902 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa790106540 0x7fa790197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:17.423 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa78ed9d640 1 -- 172.21.15.131:0/908157902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa790101470 msgr2=0x7fa790196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:17.423 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa78ed9d640 1 --2- 172.21.15.131:0/908157902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa790101470 0x7fa790196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:17.423 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa78ed9d640 1 -- 172.21.15.131:0/908157902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa788009840 con 0x7fa79019b890 2024-02-05T08:23:17.423 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa787fff640 1 --2- 172.21.15.131:0/908157902 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa790106540 0x7fa790197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:23:17.423 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa78ed9d640 1 --2- 172.21.15.131:0/908157902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79019b890 0x7fa79019bd40 secure :-1 s=READY pgs=3073 cs=0 l=1 rev1=1 crypto rx=0x7fa78800ec50 tx=0x7fa78800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:23:17.423 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa7877fe640 1 -- 172.21.15.131:0/908157902 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa788003b70 con 0x7fa79019b890 2024-02-05T08:23:17.423 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa794d7a640 1 -- 172.21.15.131:0/908157902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa778009560 con 0x7fa79019b890 2024-02-05T08:23:17.423 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa7877fe640 1 -- 172.21.15.131:0/908157902 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fa78800ce70 con 0x7fa79019b890 2024-02-05T08:23:17.423 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa7877fe640 1 -- 172.21.15.131:0/908157902 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa788010690 con 0x7fa79019b890 2024-02-05T08:23:17.424 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.420+0000 7fa794d7a640 1 -- 172.21.15.131:0/908157902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7901a3e10 con 0x7fa79019b890 2024-02-05T08:23:17.424 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.421+0000 7fa794d7a640 1 -- 172.21.15.131:0/908157902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa790068560 con 0x7fa79019b890 2024-02-05T08:23:17.424 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.422+0000 7fa7877fe640 1 -- 172.21.15.131:0/908157902 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fa788004020 con 0x7fa79019b890 2024-02-05T08:23:17.424 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.422+0000 7fa7877fe640 1 --2- 172.21.15.131:0/908157902 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa77c06d9f0 0x7fa77c06feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:17.425 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.422+0000 7fa7877fe640 1 -- 172.21.15.131:0/908157902 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(766..766 src has 254..766) v4 ==== 7801+0+0 (secure 0 0 0) 0x7fa788014070 con 0x7fa79019b890 2024-02-05T08:23:17.425 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.422+0000 7fa78e59c640 1 --2- 172.21.15.131:0/908157902 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa77c06d9f0 0x7fa77c06feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:17.425 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.423+0000 7fa78e59c640 1 --2- 172.21.15.131:0/908157902 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa77c06d9f0 0x7fa77c06feb0 secure :-1 s=READY pgs=4063 cs=0 l=1 rev1=1 crypto rx=0x7fa780005fd0 tx=0x7fa78000b390 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:23:17.427 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.427+0000 7fa7877fe640 1 -- 172.21.15.131:0/908157902 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa78805af60 con 0x7fa79019b890 2024-02-05T08:23:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:17 smithi157 ceph-mon[33345]: pgmap v1678: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:17 smithi157 ceph-mon[33345]: osdmap e765: 8 total, 8 up, 8 in 2024-02-05T08:23:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:17 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1093003226' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pg_num": 12}]: dispatch 2024-02-05T08:23:17.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:17 smithi131 ceph-mon[41654]: pgmap v1678: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:17.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:17 smithi131 ceph-mon[41654]: osdmap e765: 8 total, 8 up, 8 in 2024-02-05T08:23:17.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:17 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1093003226' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pg_num": 12}]: dispatch 2024-02-05T08:23:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:17 smithi131 ceph-mon[29507]: pgmap v1678: 196 pgs: 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:17 smithi131 ceph-mon[29507]: osdmap e765: 8 total, 8 up, 8 in 2024-02-05T08:23:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:17 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1093003226' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pg_num": 12}]: dispatch 2024-02-05T08:23:17.604 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:17.604+0000 7fa794d7a640 1 -- 172.21.15.131:0/908157902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fa790101e20 con 0x7fa79019b890 2024-02-05T08:23:18.224 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:18.223+0000 7fa7877fe640 1 -- 172.21.15.131:0/908157902 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb v767) v1 ==== 223+0+0 (secure 0 0 0) 0x7fa78805aaf0 con 0x7fa79019b890 2024-02-05T08:23:18.341 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:18.341+0000 7fa794d7a640 1 -- 172.21.15.131:0/908157902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fa790102030 con 0x7fa79019b890 2024-02-05T08:23:18.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1093003226' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pg_num": 12}]': finished 2024-02-05T08:23:18.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:18 smithi131 ceph-mon[41654]: osdmap e766: 8 total, 8 up, 8 in 2024-02-05T08:23:18.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1093003226' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pg_num": 12}]: dispatch 2024-02-05T08:23:18.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:18 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/908157902' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:23:18.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1093003226' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pg_num": 12}]': finished 2024-02-05T08:23:18.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:18 smithi131 ceph-mon[29507]: osdmap e766: 8 total, 8 up, 8 in 2024-02-05T08:23:18.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1093003226' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pg_num": 12}]: dispatch 2024-02-05T08:23:18.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:18 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/908157902' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:23:18.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1093003226' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pg_num": 12}]': finished 2024-02-05T08:23:18.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:18 smithi157 ceph-mon[33345]: osdmap e766: 8 total, 8 up, 8 in 2024-02-05T08:23:18.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1093003226' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pg_num": 12}]: dispatch 2024-02-05T08:23:18.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:18 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/908157902' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:23:19.229 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.229+0000 7fa7877fe640 1 -- 172.21.15.131:0/908157902 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb v768) v1 ==== 223+0+0 (secure 0 0 0) 0x7fa78805acd0 con 0x7fa79019b890 2024-02-05T08:23:19.230 INFO:tasks.workunit.client.0.smithi131.stderr:set-quota max_objects = 10 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb 2024-02-05T08:23:19.234 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.233+0000 7fa794d7a640 1 -- 172.21.15.131:0/908157902 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa77c06d9f0 msgr2=0x7fa77c06feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:19.234 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.233+0000 7fa794d7a640 1 --2- 172.21.15.131:0/908157902 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa77c06d9f0 0x7fa77c06feb0 secure :-1 s=READY pgs=4063 cs=0 l=1 rev1=1 crypto rx=0x7fa780005fd0 tx=0x7fa78000b390 comp rx=0 tx=0).stop 2024-02-05T08:23:19.234 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.233+0000 7fa794d7a640 1 -- 172.21.15.131:0/908157902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79019b890 msgr2=0x7fa79019bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:19.234 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.233+0000 7fa794d7a640 1 --2- 172.21.15.131:0/908157902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79019b890 0x7fa79019bd40 secure :-1 s=READY pgs=3073 cs=0 l=1 rev1=1 crypto rx=0x7fa78800ec50 tx=0x7fa78800c6a0 comp rx=0 tx=0).stop 2024-02-05T08:23:19.234 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.233+0000 7fa794d7a640 1 -- 172.21.15.131:0/908157902 shutdown_connections 2024-02-05T08:23:19.234 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.233+0000 7fa794d7a640 1 --2- 172.21.15.131:0/908157902 >> v2:172.21.15.131:6800/2614238408 conn(0x7fa77c06d9f0 0x7fa77c06feb0 unknown :-1 s=CLOSED pgs=4063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:19.234 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.233+0000 7fa794d7a640 1 --2- 172.21.15.131:0/908157902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa79019b890 0x7fa79019bd40 unknown :-1 s=CLOSED pgs=3073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:19.234 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.233+0000 7fa794d7a640 1 --2- 172.21.15.131:0/908157902 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fa790106540 0x7fa790197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:19.234 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.233+0000 7fa794d7a640 1 --2- 172.21.15.131:0/908157902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa790101470 0x7fa790196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:19.234 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.233+0000 7fa794d7a640 1 -- 172.21.15.131:0/908157902 >> 172.21.15.131:0/908157902 conn(0x7fa7900fcd30 msgr2=0x7fa7900fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:23:19.235 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.235+0000 7fa794d7a640 1 -- 172.21.15.131:0/908157902 shutdown_connections 2024-02-05T08:23:19.235 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.235+0000 7fa794d7a640 1 -- 172.21.15.131:0/908157902 wait complete. 2024-02-05T08:23:19.247 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool application enable b9122de8-c2e1-4670-8261-fe325aa3ccfb rados 2024-02-05T08:23:19.338 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.335+0000 7efe0fe84640 1 -- 172.21.15.131:0/3781791827 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7efe080ffba0 msgr2=0x7efe08100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:19.338 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.335+0000 7efe0fe84640 1 --2- 172.21.15.131:0/3781791827 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7efe080ffba0 0x7efe08100020 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009890 tx=0x7efdf802bb70 comp rx=0 tx=0).stop 2024-02-05T08:23:19.338 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.335+0000 7efe0fe84640 1 -- 172.21.15.131:0/3781791827 shutdown_connections 2024-02-05T08:23:19.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.335+0000 7efe0fe84640 1 --2- 172.21.15.131:0/3781791827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe08100560 0x7efe0810e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:19.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.335+0000 7efe0fe84640 1 --2- 172.21.15.131:0/3781791827 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7efe080ffba0 0x7efe08100020 unknown :-1 s=CLOSED pgs=3060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:19.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.335+0000 7efe0fe84640 1 --2- 172.21.15.131:0/3781791827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe08106280 0x7efe08106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:19.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.335+0000 7efe0fe84640 1 -- 172.21.15.131:0/3781791827 >> 172.21.15.131:0/3781791827 conn(0x7efe08075410 msgr2=0x7efe08075820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:23:19.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.335+0000 7efe0fe84640 1 -- 172.21.15.131:0/3781791827 shutdown_connections 2024-02-05T08:23:19.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.335+0000 7efe0fe84640 1 -- 172.21.15.131:0/3781791827 wait complete. 2024-02-05T08:23:19.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.335+0000 7efe0fe84640 1 Processor -- start 2024-02-05T08:23:19.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.335+0000 7efe0fe84640 1 -- start start 2024-02-05T08:23:19.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0fe84640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7efe080ffba0 0x7efe08196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:19.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0fe84640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe08100560 0x7efe08197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:19.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0fe84640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe08106280 0x7efe0819b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:19.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0fe84640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0819be40 con 0x7efe08100560 2024-02-05T08:23:19.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0fe84640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0819bfb0 con 0x7efe08106280 2024-02-05T08:23:19.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0fe84640 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0819c170 con 0x7efe080ffba0 2024-02-05T08:23:19.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0dc20640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7efe080ffba0 0x7efe08196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:19.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0d41f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe08100560 0x7efe08197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:19.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0dc20640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7efe080ffba0 0x7efe08196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:33576/0 (socket says 172.21.15.131:33576) 2024-02-05T08:23:19.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0dc20640 1 -- 172.21.15.131:0/3919572560 learned_addr learned my addr 172.21.15.131:0/3919572560 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T08:23:19.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0e421640 1 --2- 172.21.15.131:0/3919572560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe08106280 0x7efe0819b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:19.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0d41f640 1 -- 172.21.15.131:0/3919572560 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7efe080ffba0 msgr2=0x7efe08196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:19.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0d41f640 1 --2- 172.21.15.131:0/3919572560 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7efe080ffba0 0x7efe08196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:19.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0d41f640 1 -- 172.21.15.131:0/3919572560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe08106280 msgr2=0x7efe0819b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:19.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0d41f640 1 --2- 172.21.15.131:0/3919572560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe08106280 0x7efe0819b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:19.341 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0d41f640 1 -- 172.21.15.131:0/3919572560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe00009560 con 0x7efe08100560 2024-02-05T08:23:19.341 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0e421640 1 --2- 172.21.15.131:0/3919572560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe08106280 0x7efe0819b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T08:23:19.341 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0d41f640 1 --2- 172.21.15.131:0/3919572560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe08100560 0x7efe08197510 secure :-1 s=READY pgs=3074 cs=0 l=1 rev1=1 crypto rx=0x7efdf802c0b0 tx=0x7efdf80344c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:23:19.341 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efdfeffd640 1 -- 172.21.15.131:0/3919572560 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf802c710 con 0x7efe08100560 2024-02-05T08:23:19.341 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0fe84640 1 -- 172.21.15.131:0/3919572560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf8009490 con 0x7efe08100560 2024-02-05T08:23:19.341 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efdfeffd640 1 -- 172.21.15.131:0/3919572560 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7efdf802ce90 con 0x7efe08100560 2024-02-05T08:23:19.341 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efdfeffd640 1 -- 172.21.15.131:0/3919572560 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf80324a0 con 0x7efe08100560 2024-02-05T08:23:19.341 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.336+0000 7efe0fe84640 1 -- 172.21.15.131:0/3919572560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe081a3af0 con 0x7efe08100560 2024-02-05T08:23:19.341 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.338+0000 7efe0fe84640 1 -- 172.21.15.131:0/3919572560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe08068560 con 0x7efe08100560 2024-02-05T08:23:19.341 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.338+0000 7efdfeffd640 1 -- 172.21.15.131:0/3919572560 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7efdf803b040 con 0x7efe08100560 2024-02-05T08:23:19.341 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.338+0000 7efdfeffd640 1 --2- 172.21.15.131:0/3919572560 >> v2:172.21.15.131:6800/2614238408 conn(0x7efdf406d9f0 0x7efdf406feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:19.341 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.338+0000 7efe0dc20640 1 --2- 172.21.15.131:0/3919572560 >> v2:172.21.15.131:6800/2614238408 conn(0x7efdf406d9f0 0x7efdf406feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:19.342 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.338+0000 7efdfeffd640 1 -- 172.21.15.131:0/3919572560 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(768..768 src has 254..768) v4 ==== 7801+0+0 (secure 0 0 0) 0x7efdf80af490 con 0x7efe08100560 2024-02-05T08:23:19.342 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.339+0000 7efe0dc20640 1 --2- 172.21.15.131:0/3919572560 >> v2:172.21.15.131:6800/2614238408 conn(0x7efdf406d9f0 0x7efdf406feb0 secure :-1 s=READY pgs=4064 cs=0 l=1 rev1=1 crypto rx=0x7efe04009270 tx=0x7efe04009700 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:23:19.343 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.343+0000 7efdfeffd640 1 -- 172.21.15.131:0/3919572560 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efdf807c360 con 0x7efe08100560 2024-02-05T08:23:19.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:19 smithi131 ceph-mon[41654]: pgmap v1681: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 383 B/s rd, 0 op/s 2024-02-05T08:23:19.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:19 smithi131 ceph-mon[41654]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:23:19.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/908157902' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:23:19.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:19 smithi131 ceph-mon[41654]: osdmap e767: 8 total, 8 up, 8 in 2024-02-05T08:23:19.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:19 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/908157902' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:23:19.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:19 smithi131 ceph-mon[29507]: pgmap v1681: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 383 B/s rd, 0 op/s 2024-02-05T08:23:19.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:19 smithi131 ceph-mon[29507]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:23:19.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/908157902' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:23:19.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:19 smithi131 ceph-mon[29507]: osdmap e767: 8 total, 8 up, 8 in 2024-02-05T08:23:19.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:19 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/908157902' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:23:19.521 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:19.521+0000 7efe0fe84640 1 -- 172.21.15.131:0/3919572560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "app": "rados"} v 0) v1 -- 0x7efe08110a00 con 0x7efe08100560 2024-02-05T08:23:19.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:19 smithi157 ceph-mon[33345]: pgmap v1681: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 383 B/s rd, 0 op/s 2024-02-05T08:23:19.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:19 smithi157 ceph-mon[33345]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:23:19.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/908157902' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:23:19.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:19 smithi157 ceph-mon[33345]: osdmap e767: 8 total, 8 up, 8 in 2024-02-05T08:23:19.721 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:19 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/908157902' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:23:20.235 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:20.235+0000 7efdfeffd640 1 -- 172.21.15.131:0/3919572560 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "app": "rados"}]=0 enabled application 'rados' on pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' v769) v1 ==== 213+0+0 (secure 0 0 0) 0x7efdf807bef0 con 0x7efe08100560 2024-02-05T08:23:20.344 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:20.345+0000 7efe0fe84640 1 -- 172.21.15.131:0/3919572560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "app": "rados"} v 0) v1 -- 0x7efe08110cb0 con 0x7efe08100560 2024-02-05T08:23:20.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:20 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/908157902' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:23:20.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:20 smithi157 ceph-mon[33345]: osdmap e768: 8 total, 8 up, 8 in 2024-02-05T08:23:20.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:20 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3919572560' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "app": "rados"}]: dispatch 2024-02-05T08:23:20.471 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:23:20 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:23:20.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:20 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/908157902' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:23:20.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:20 smithi131 ceph-mon[41654]: osdmap e768: 8 total, 8 up, 8 in 2024-02-05T08:23:20.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:20 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3919572560' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "app": "rados"}]: dispatch 2024-02-05T08:23:20.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:20 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/908157902' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:23:20.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:20 smithi131 ceph-mon[29507]: osdmap e768: 8 total, 8 up, 8 in 2024-02-05T08:23:20.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:20 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3919572560' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "app": "rados"}]: dispatch 2024-02-05T08:23:21.237 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:21.237+0000 7efdfeffd640 1 -- 172.21.15.131:0/3919572560 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "app": "rados"}]=0 enabled application 'rados' on pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' v770) v1 ==== 213+0+0 (secure 0 0 0) 0x7efdf804b0d0 con 0x7efe08100560 2024-02-05T08:23:21.238 INFO:tasks.workunit.client.0.smithi131.stderr:enabled application 'rados' on pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' 2024-02-05T08:23:21.242 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:21.241+0000 7efe0fe84640 1 -- 172.21.15.131:0/3919572560 >> v2:172.21.15.131:6800/2614238408 conn(0x7efdf406d9f0 msgr2=0x7efdf406feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:21.242 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:21.241+0000 7efe0fe84640 1 --2- 172.21.15.131:0/3919572560 >> v2:172.21.15.131:6800/2614238408 conn(0x7efdf406d9f0 0x7efdf406feb0 secure :-1 s=READY pgs=4064 cs=0 l=1 rev1=1 crypto rx=0x7efe04009270 tx=0x7efe04009700 comp rx=0 tx=0).stop 2024-02-05T08:23:21.242 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:21.241+0000 7efe0fe84640 1 -- 172.21.15.131:0/3919572560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe08100560 msgr2=0x7efe08197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:21.242 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:21.241+0000 7efe0fe84640 1 --2- 172.21.15.131:0/3919572560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe08100560 0x7efe08197510 secure :-1 s=READY pgs=3074 cs=0 l=1 rev1=1 crypto rx=0x7efdf802c0b0 tx=0x7efdf80344c0 comp rx=0 tx=0).stop 2024-02-05T08:23:21.242 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:21.242+0000 7efe0fe84640 1 -- 172.21.15.131:0/3919572560 shutdown_connections 2024-02-05T08:23:21.242 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:21.242+0000 7efe0fe84640 1 --2- 172.21.15.131:0/3919572560 >> v2:172.21.15.131:6800/2614238408 conn(0x7efdf406d9f0 0x7efdf406feb0 unknown :-1 s=CLOSED pgs=4064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:21.242 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:21.242+0000 7efe0fe84640 1 --2- 172.21.15.131:0/3919572560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe08106280 0x7efe0819b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:21.242 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:21.242+0000 7efe0fe84640 1 --2- 172.21.15.131:0/3919572560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe08100560 0x7efe08197510 unknown :-1 s=CLOSED pgs=3074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:21.242 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:21.242+0000 7efe0fe84640 1 --2- 172.21.15.131:0/3919572560 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7efe080ffba0 0x7efe08196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:21.242 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:21.242+0000 7efe0fe84640 1 -- 172.21.15.131:0/3919572560 >> 172.21.15.131:0/3919572560 conn(0x7efe08075410 msgr2=0x7efe080fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:23:21.243 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:21.242+0000 7efe0fe84640 1 -- 172.21.15.131:0/3919572560 shutdown_connections 2024-02-05T08:23:21.243 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:21.242+0000 7efe0fe84640 1 -- 172.21.15.131:0/3919572560 wait complete. 2024-02-05T08:23:21.255 INFO:tasks.workunit.client.0.smithi131.stderr:++ seq 1 10 2024-02-05T08:23:21.272 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:23:21.272 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p b9122de8-c2e1-4670-8261-fe325aa3ccfb put obj1 /etc/passwd 2024-02-05T08:23:21.310 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:23:21.310 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p b9122de8-c2e1-4670-8261-fe325aa3ccfb put obj2 /etc/passwd 2024-02-05T08:23:21.345 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:23:21.345 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p b9122de8-c2e1-4670-8261-fe325aa3ccfb put obj3 /etc/passwd 2024-02-05T08:23:21.381 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:23:21.381 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p b9122de8-c2e1-4670-8261-fe325aa3ccfb put obj4 /etc/passwd 2024-02-05T08:23:21.416 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:23:21.416 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p b9122de8-c2e1-4670-8261-fe325aa3ccfb put obj5 /etc/passwd 2024-02-05T08:23:21.451 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:23:21.451 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p b9122de8-c2e1-4670-8261-fe325aa3ccfb put obj6 /etc/passwd 2024-02-05T08:23:21.485 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:23:21.485 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p b9122de8-c2e1-4670-8261-fe325aa3ccfb put obj7 /etc/passwd 2024-02-05T08:23:21.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:21 smithi131 ceph-mon[41654]: pgmap v1684: 176 pgs: 2 creating+activating, 174 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:23:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:21 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3919572560' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "app": "rados"}]': finished 2024-02-05T08:23:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:21 smithi131 ceph-mon[41654]: osdmap e769: 8 total, 8 up, 8 in 2024-02-05T08:23:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:21 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3919572560' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "app": "rados"}]: dispatch 2024-02-05T08:23:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:21 smithi131 ceph-mon[29507]: pgmap v1684: 176 pgs: 2 creating+activating, 174 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:23:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:21 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3919572560' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "app": "rados"}]': finished 2024-02-05T08:23:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:21 smithi131 ceph-mon[29507]: osdmap e769: 8 total, 8 up, 8 in 2024-02-05T08:23:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:21 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3919572560' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "app": "rados"}]: dispatch 2024-02-05T08:23:21.519 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:23:21.520 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p b9122de8-c2e1-4670-8261-fe325aa3ccfb put obj8 /etc/passwd 2024-02-05T08:23:21.554 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:23:21.554 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p b9122de8-c2e1-4670-8261-fe325aa3ccfb put obj9 /etc/passwd 2024-02-05T08:23:21.592 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:23:21.593 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p b9122de8-c2e1-4670-8261-fe325aa3ccfb put obj10 /etc/passwd 2024-02-05T08:23:21.624 INFO:tasks.workunit.client.0.smithi131.stderr:+ sleep 30 2024-02-05T08:23:21.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:21 smithi157 ceph-mon[33345]: pgmap v1684: 176 pgs: 2 creating+activating, 174 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T08:23:21.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:21 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:21.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3919572560' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "app": "rados"}]': finished 2024-02-05T08:23:21.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:21 smithi157 ceph-mon[33345]: osdmap e769: 8 total, 8 up, 8 in 2024-02-05T08:23:21.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:21 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3919572560' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "app": "rados"}]: dispatch 2024-02-05T08:23:22.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:22 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3919572560' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "app": "rados"}]': finished 2024-02-05T08:23:22.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:22 smithi131 ceph-mon[41654]: osdmap e770: 8 total, 8 up, 8 in 2024-02-05T08:23:22.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:22 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3919572560' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "app": "rados"}]': finished 2024-02-05T08:23:22.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:22 smithi131 ceph-mon[29507]: osdmap e770: 8 total, 8 up, 8 in 2024-02-05T08:23:22.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:22 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3919572560' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "app": "rados"}]': finished 2024-02-05T08:23:22.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:22 smithi157 ceph-mon[33345]: osdmap e770: 8 total, 8 up, 8 in 2024-02-05T08:23:23.250 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:23:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:23:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:23:23.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:23 smithi131 ceph-mon[29507]: pgmap v1687: 176 pgs: 2 creating+activating, 174 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:23.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:23 smithi131 ceph-mon[41654]: pgmap v1687: 176 pgs: 2 creating+activating, 174 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:23.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:23 smithi157 ceph-mon[33345]: pgmap v1687: 176 pgs: 2 creating+activating, 174 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:25 smithi131 ceph-mon[41654]: pgmap v1688: 176 pgs: 176 active+clean; 695 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 21 KiB/s wr, 3 op/s 2024-02-05T08:23:25.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:25 smithi131 ceph-mon[29507]: pgmap v1688: 176 pgs: 176 active+clean; 695 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 21 KiB/s wr, 3 op/s 2024-02-05T08:23:25.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:25 smithi157 ceph-mon[33345]: pgmap v1688: 176 pgs: 176 active+clean; 695 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 21 KiB/s wr, 3 op/s 2024-02-05T08:23:27.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:27 smithi131 ceph-mon[41654]: pgmap v1689: 176 pgs: 176 active+clean; 695 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 758 B/s rd, 18 KiB/s wr, 2 op/s 2024-02-05T08:23:27.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:27 smithi131 ceph-mon[41654]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:23:27.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:27 smithi131 ceph-mon[41654]: pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' is full (reached quota's max_objects: 10) 2024-02-05T08:23:27.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:27 smithi131 ceph-mon[41654]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:23:27.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:27 smithi131 ceph-mon[41654]: osdmap e771: 8 total, 8 up, 8 in 2024-02-05T08:23:27.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:27 smithi131 ceph-mon[29507]: pgmap v1689: 176 pgs: 176 active+clean; 695 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 758 B/s rd, 18 KiB/s wr, 2 op/s 2024-02-05T08:23:27.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:27 smithi131 ceph-mon[29507]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:23:27.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:27 smithi131 ceph-mon[29507]: pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' is full (reached quota's max_objects: 10) 2024-02-05T08:23:27.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:27 smithi131 ceph-mon[29507]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:23:27.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:27 smithi131 ceph-mon[29507]: osdmap e771: 8 total, 8 up, 8 in 2024-02-05T08:23:27.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:27 smithi157 ceph-mon[33345]: pgmap v1689: 176 pgs: 176 active+clean; 695 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 758 B/s rd, 18 KiB/s wr, 2 op/s 2024-02-05T08:23:27.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:27 smithi157 ceph-mon[33345]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:23:27.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:27 smithi157 ceph-mon[33345]: pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' is full (reached quota's max_objects: 10) 2024-02-05T08:23:27.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:27 smithi157 ceph-mon[33345]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:23:27.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:27 smithi157 ceph-mon[33345]: osdmap e771: 8 total, 8 up, 8 in 2024-02-05T08:23:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:29 smithi131 ceph-mon[41654]: pgmap v1691: 176 pgs: 176 active+clean; 695 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 661 B/s rd, 15 KiB/s wr, 1 op/s 2024-02-05T08:23:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:29 smithi131 ceph-mon[29507]: pgmap v1691: 176 pgs: 176 active+clean; 695 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 661 B/s rd, 15 KiB/s wr, 1 op/s 2024-02-05T08:23:29.522 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:29 smithi157 ceph-mon[33345]: pgmap v1691: 176 pgs: 176 active+clean; 695 KiB data, 858 MiB used, 714 GiB / 715 GiB avail; 661 B/s rd, 15 KiB/s wr, 1 op/s 2024-02-05T08:23:30.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:23:30 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:23:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:31 smithi131 ceph-mon[41654]: pgmap v1692: 176 pgs: 176 active+clean; 695 KiB data, 856 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-02-05T08:23:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:31 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:31 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:31 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:23:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:31 smithi131 ceph-mon[29507]: pgmap v1692: 176 pgs: 176 active+clean; 695 KiB data, 856 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-02-05T08:23:31.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:31 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:31.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:31 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:31.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:31 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:23:31.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:31 smithi157 ceph-mon[33345]: pgmap v1692: 176 pgs: 176 active+clean; 695 KiB data, 856 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-02-05T08:23:31.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:31 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:31.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:31 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:31.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:31 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:23:33.264 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:23:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:23:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:23:33.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:33 smithi131 ceph-mon[41654]: pgmap v1693: 176 pgs: 176 active+clean; 695 KiB data, 856 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-05T08:23:33.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:33 smithi131 ceph-mon[29507]: pgmap v1693: 176 pgs: 176 active+clean; 695 KiB data, 856 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-05T08:23:33.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:33 smithi157 ceph-mon[33345]: pgmap v1693: 176 pgs: 176 active+clean; 695 KiB data, 856 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-05T08:23:35.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:35 smithi157 ceph-mon[33345]: pgmap v1694: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:23:35.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:35 smithi131 ceph-mon[29507]: pgmap v1694: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:23:35.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:35 smithi131 ceph-mon[41654]: pgmap v1694: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:23:37.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:37 smithi157 ceph-mon[33345]: pgmap v1695: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:23:37.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:37 smithi131 ceph-mon[41654]: pgmap v1695: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:23:37.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:37 smithi131 ceph-mon[29507]: pgmap v1695: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:23:39.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:39 smithi157 ceph-mon[33345]: pgmap v1696: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 932 B/s rd, 0 op/s 2024-02-05T08:23:39.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:39 smithi131 ceph-mon[41654]: pgmap v1696: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 932 B/s rd, 0 op/s 2024-02-05T08:23:39.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:39 smithi131 ceph-mon[29507]: pgmap v1696: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 932 B/s rd, 0 op/s 2024-02-05T08:23:40.472 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:23:40 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:23:41.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:41 smithi157 ceph-mon[33345]: pgmap v1697: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:41.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:41 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:41.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:41 smithi131 ceph-mon[41654]: pgmap v1697: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:41.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:41 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:41.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:41 smithi131 ceph-mon[29507]: pgmap v1697: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:41.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:41 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:23:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:23:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:23:43.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:43 smithi157 ceph-mon[33345]: pgmap v1698: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:23:43.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:43 smithi131 ceph-mon[41654]: pgmap v1698: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:23:43.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:43 smithi131 ceph-mon[29507]: pgmap v1698: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:23:45.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:45 smithi157 ceph-mon[33345]: pgmap v1699: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:45.766 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:45 smithi131 ceph-mon[29507]: pgmap v1699: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:45.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:45 smithi131 ceph-mon[41654]: pgmap v1699: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:46.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:46 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:23:46.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:46 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:23:46.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:46 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:23:47.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:47 smithi157 ceph-mon[33345]: pgmap v1700: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:47.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:47 smithi131 ceph-mon[41654]: pgmap v1700: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:47.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:47 smithi131 ceph-mon[29507]: pgmap v1700: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:48.720 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:48 smithi157 ceph-mon[33345]: pgmap v1701: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:48.766 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:48 smithi131 ceph-mon[41654]: pgmap v1701: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:48.767 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:48 smithi131 ceph-mon[29507]: pgmap v1701: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:50.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:23:50 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:23:51.373 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:51 smithi131 ceph-mon[29507]: pgmap v1702: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:51.373 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:51 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:51.373 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:51 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:23:51.373 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:51 smithi131 ceph-mon[41654]: pgmap v1702: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:51.374 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:51 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:51.374 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:51 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:23:51.408 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:51 smithi157 ceph-mon[33345]: pgmap v1702: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:51.408 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:51 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:23:51.409 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:51 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:23:51.625 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=158598 2024-02-05T08:23:51.625 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool set-quota b9122de8-c2e1-4670-8261-fe325aa3ccfb max_objects 100 2024-02-05T08:23:51.625 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p b9122de8-c2e1-4670-8261-fe325aa3ccfb put onemore /etc/passwd 2024-02-05T08:23:51.723 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.720+0000 7f60a6fc7640 1 -- 172.21.15.131:0/1812669855 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f60a0105450 msgr2=0x7f60a010d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:51.723 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.720+0000 7f60a6fc7640 1 --2- 172.21.15.131:0/1812669855 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f60a0105450 0x7f60a010d020 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 crypto rx=0x7f6098009ca0 tx=0x7f609802c0f0 comp rx=0 tx=0).stop 2024-02-05T08:23:51.723 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.720+0000 7f60a6fc7640 1 -- 172.21.15.131:0/1812669855 shutdown_connections 2024-02-05T08:23:51.723 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.720+0000 7f60a6fc7640 1 --2- 172.21.15.131:0/1812669855 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f60a0105450 0x7f60a010d020 unknown :-1 s=CLOSED pgs=3087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:51.723 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.720+0000 7f60a6fc7640 1 --2- 172.21.15.131:0/1812669855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60a0100d30 0x7f60a0104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:51.724 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.720+0000 7f60a6fc7640 1 --2- 172.21.15.131:0/1812669855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a0100380 0x7f60a0100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:51.724 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.720+0000 7f60a6fc7640 1 -- 172.21.15.131:0/1812669855 >> 172.21.15.131:0/1812669855 conn(0x7f60a0077220 msgr2=0x7f60a0077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:23:51.724 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.720+0000 7f60a6fc7640 1 -- 172.21.15.131:0/1812669855 shutdown_connections 2024-02-05T08:23:51.724 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.720+0000 7f60a6fc7640 1 -- 172.21.15.131:0/1812669855 wait complete. 2024-02-05T08:23:51.724 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a6fc7640 1 Processor -- start 2024-02-05T08:23:51.724 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a6fc7640 1 -- start start 2024-02-05T08:23:51.724 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a6fc7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a0100380 0x7f60a0196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:51.724 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a6fc7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60a0105450 0x7f60a01974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:51.724 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a6fc7640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f60a019b800 0x7f60a019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:51.724 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a6fc7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a019c1f0 con 0x7f60a0100380 2024-02-05T08:23:51.724 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a6fc7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a019c360 con 0x7f60a0105450 2024-02-05T08:23:51.724 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a6fc7640 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a019c4d0 con 0x7f60a019b800 2024-02-05T08:23:51.724 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a5564640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f60a019b800 0x7f60a019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:51.724 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a4d63640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a0100380 0x7f60a0196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:51.725 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a5564640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f60a019b800 0x7f60a019bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:44344/0 (socket says 172.21.15.131:44344) 2024-02-05T08:23:51.725 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a5564640 1 -- 172.21.15.131:0/1044577557 learned_addr learned my addr 172.21.15.131:0/1044577557 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T08:23:51.725 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f609ffff640 1 --2- 172.21.15.131:0/1044577557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60a0105450 0x7f60a01974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:51.725 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a5564640 1 -- 172.21.15.131:0/1044577557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60a0105450 msgr2=0x7f60a01974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:51.725 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a5564640 1 --2- 172.21.15.131:0/1044577557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60a0105450 0x7f60a01974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:51.725 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a5564640 1 -- 172.21.15.131:0/1044577557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a0100380 msgr2=0x7f60a0196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:51.725 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a5564640 1 --2- 172.21.15.131:0/1044577557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a0100380 0x7f60a0196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:51.725 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a5564640 1 -- 172.21.15.131:0/1044577557 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6094009490 con 0x7f60a019b800 2024-02-05T08:23:51.725 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.721+0000 7f60a4d63640 1 --2- 172.21.15.131:0/1044577557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a0100380 0x7f60a0196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:23:51.725 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.722+0000 7f60a5564640 1 --2- 172.21.15.131:0/1044577557 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f60a019b800 0x7f60a019bcb0 secure :-1 s=READY pgs=3088 cs=0 l=1 rev1=1 crypto rx=0x7f6098005e30 tx=0x7f6098034760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:23:51.725 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.722+0000 7f609dffb640 1 -- 172.21.15.131:0/1044577557 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6098034a60 con 0x7f60a019b800 2024-02-05T08:23:51.725 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.722+0000 7f609dffb640 1 -- 172.21.15.131:0/1044577557 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f6098003e20 con 0x7f60a019b800 2024-02-05T08:23:51.725 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.722+0000 7f60a6fc7640 1 -- 172.21.15.131:0/1044577557 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6098009900 con 0x7f60a019b800 2024-02-05T08:23:51.725 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.722+0000 7f609dffb640 1 -- 172.21.15.131:0/1044577557 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6098004630 con 0x7f60a019b800 2024-02-05T08:23:51.725 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.722+0000 7f60a6fc7640 1 -- 172.21.15.131:0/1044577557 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a01a3d40 con 0x7f60a019b800 2024-02-05T08:23:51.726 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.723+0000 7f60a6fc7640 1 -- 172.21.15.131:0/1044577557 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a0067880 con 0x7f60a019b800 2024-02-05T08:23:51.727 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.724+0000 7f609dffb640 1 -- 172.21.15.131:0/1044577557 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f609803b040 con 0x7f60a019b800 2024-02-05T08:23:51.727 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.724+0000 7f609dffb640 1 --2- 172.21.15.131:0/1044577557 >> v2:172.21.15.131:6800/2614238408 conn(0x7f609006d9f0 0x7f609006feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:51.727 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.724+0000 7f60a4d63640 1 --2- 172.21.15.131:0/1044577557 >> v2:172.21.15.131:6800/2614238408 conn(0x7f609006d9f0 0x7f609006feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:51.727 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.724+0000 7f609dffb640 1 -- 172.21.15.131:0/1044577557 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(771..771 src has 254..771) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f60980afeb0 con 0x7f60a019b800 2024-02-05T08:23:51.727 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.725+0000 7f609dffb640 1 -- 172.21.15.131:0/1044577557 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=772}) v3 -- 0x7f6090078f40 con 0x7f60a019b800 2024-02-05T08:23:51.727 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.725+0000 7f60a4d63640 1 --2- 172.21.15.131:0/1044577557 >> v2:172.21.15.131:6800/2614238408 conn(0x7f609006d9f0 0x7f609006feb0 secure :-1 s=READY pgs=4076 cs=0 l=1 rev1=1 crypto rx=0x7f6094011630 tx=0x7f6094005f90 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:23:51.730 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.730+0000 7f609dffb640 1 -- 172.21.15.131:0/1044577557 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f609807cd70 con 0x7f60a019b800 2024-02-05T08:23:51.928 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:51.928+0000 7f60a6fc7640 1 -- 172.21.15.131:0/1044577557 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f60a0100d30 con 0x7f60a019b800 2024-02-05T08:23:52.057 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:52.053+0000 7f609dffb640 1 -- 172.21.15.131:0/1044577557 <== mon.2 v2:172.21.15.131:3301/0 7 ==== osd_map(772..772 src has 254..772) v4 ==== 628+0+0 (secure 0 0 0) 0x7f6098081040 con 0x7f60a019b800 2024-02-05T08:23:52.057 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:52.053+0000 7f609dffb640 1 -- 172.21.15.131:0/1044577557 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=773}) v3 -- 0x7f6090079a70 con 0x7f60a019b800 2024-02-05T08:23:52.057 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:52.055+0000 7f609dffb640 1 -- 172.21.15.131:0/1044577557 <== mon.2 v2:172.21.15.131:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb v772) v1 ==== 225+0+0 (secure 0 0 0) 0x7f609804b060 con 0x7f60a019b800 2024-02-05T08:23:52.162 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:52.161+0000 7f60a6fc7640 1 -- 172.21.15.131:0/1044577557 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f60a0100f40 con 0x7f60a019b800 2024-02-05T08:23:52.314 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:52 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1044577557' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:23:52.315 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:52 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:23:52.315 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:52 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1044577557' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:23:52.315 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:52 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:23:52.315 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:52 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1044577557' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:23:52.316 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:52 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:23:53.061 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:53.056+0000 7f609dffb640 1 -- 172.21.15.131:0/1044577557 <== mon.2 v2:172.21.15.131:3301/0 9 ==== osd_map(773..773 src has 254..773) v4 ==== 628+0+0 (secure 0 0 0) 0x7f6098085600 con 0x7f60a019b800 2024-02-05T08:23:53.061 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:53.056+0000 7f609dffb640 1 -- 172.21.15.131:0/1044577557 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=774}) v3 -- 0x7f6090079ca0 con 0x7f60a019b800 2024-02-05T08:23:53.062 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:53.061+0000 7f609dffb640 1 -- 172.21.15.131:0/1044577557 <== mon.2 v2:172.21.15.131:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb v773) v1 ==== 225+0+0 (secure 0 0 0) 0x7f6098041070 con 0x7f60a019b800 2024-02-05T08:23:53.062 INFO:tasks.workunit.client.0.smithi131.stderr:set-quota max_objects = 100 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb 2024-02-05T08:23:53.065 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:53.064+0000 7f60a6fc7640 1 -- 172.21.15.131:0/1044577557 >> v2:172.21.15.131:6800/2614238408 conn(0x7f609006d9f0 msgr2=0x7f609006feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:53.065 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:53.064+0000 7f60a6fc7640 1 --2- 172.21.15.131:0/1044577557 >> v2:172.21.15.131:6800/2614238408 conn(0x7f609006d9f0 0x7f609006feb0 secure :-1 s=READY pgs=4076 cs=0 l=1 rev1=1 crypto rx=0x7f6094011630 tx=0x7f6094005f90 comp rx=0 tx=0).stop 2024-02-05T08:23:53.065 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:53.064+0000 7f60a6fc7640 1 -- 172.21.15.131:0/1044577557 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f60a019b800 msgr2=0x7f60a019bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:53.065 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:53.064+0000 7f60a6fc7640 1 --2- 172.21.15.131:0/1044577557 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f60a019b800 0x7f60a019bcb0 secure :-1 s=READY pgs=3088 cs=0 l=1 rev1=1 crypto rx=0x7f6098005e30 tx=0x7f6098034760 comp rx=0 tx=0).stop 2024-02-05T08:23:53.065 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:53.064+0000 7f60a6fc7640 1 -- 172.21.15.131:0/1044577557 shutdown_connections 2024-02-05T08:23:53.065 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:53.064+0000 7f60a6fc7640 1 --2- 172.21.15.131:0/1044577557 >> v2:172.21.15.131:6800/2614238408 conn(0x7f609006d9f0 0x7f609006feb0 unknown :-1 s=CLOSED pgs=4076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:53.066 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:53.064+0000 7f60a6fc7640 1 --2- 172.21.15.131:0/1044577557 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f60a019b800 0x7f60a019bcb0 unknown :-1 s=CLOSED pgs=3088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:53.066 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:53.064+0000 7f60a6fc7640 1 --2- 172.21.15.131:0/1044577557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60a0105450 0x7f60a01974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:53.066 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:53.064+0000 7f60a6fc7640 1 --2- 172.21.15.131:0/1044577557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a0100380 0x7f60a0196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:53.066 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:53.064+0000 7f60a6fc7640 1 -- 172.21.15.131:0/1044577557 >> 172.21.15.131:0/1044577557 conn(0x7f60a0077220 msgr2=0x7f60a0103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:23:53.066 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:53.064+0000 7f60a6fc7640 1 -- 172.21.15.131:0/1044577557 shutdown_connections 2024-02-05T08:23:53.066 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:53.064+0000 7f60a6fc7640 1 -- 172.21.15.131:0/1044577557 wait complete. 2024-02-05T08:23:53.076 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 158598 2024-02-05T08:23:53.180 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:53 smithi157 ceph-mon[33345]: pgmap v1703: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:53.180 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]': finished 2024-02-05T08:23:53.180 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:53 smithi157 ceph-mon[33345]: osdmap e772: 8 total, 8 up, 8 in 2024-02-05T08:23:53.180 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:53 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/1044577557' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:23:53.180 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:53 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:23:53.194 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:23:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:23:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:23:53.195 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:53 smithi131 ceph-mon[41654]: pgmap v1703: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:53.195 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]': finished 2024-02-05T08:23:53.195 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:53 smithi131 ceph-mon[41654]: osdmap e772: 8 total, 8 up, 8 in 2024-02-05T08:23:53.195 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:53 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/1044577557' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:23:53.195 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:53 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:23:53.195 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:53 smithi131 ceph-mon[29507]: pgmap v1703: 176 pgs: 176 active+clean; 695 KiB data, 855 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:53.195 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]': finished 2024-02-05T08:23:53.196 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:53 smithi131 ceph-mon[29507]: osdmap e772: 8 total, 8 up, 8 in 2024-02-05T08:23:53.196 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:53 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/1044577557' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:23:53.196 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:53 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:23:54.427 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:54 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]': finished 2024-02-05T08:23:54.427 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:54 smithi131 ceph-mon[41654]: osdmap e773: 8 total, 8 up, 8 in 2024-02-05T08:23:54.427 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:54 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:54.427 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:54 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:54.427 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:54 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]': finished 2024-02-05T08:23:54.427 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:54 smithi131 ceph-mon[29507]: osdmap e773: 8 total, 8 up, 8 in 2024-02-05T08:23:54.427 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:54 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:54.427 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:54 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:54.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:54 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "100"}]': finished 2024-02-05T08:23:54.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:54 smithi157 ceph-mon[33345]: osdmap e773: 8 total, 8 up, 8 in 2024-02-05T08:23:54.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:54 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:54.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:54 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:55 smithi157 ceph-mon[33345]: pgmap v1706: 176 pgs: 176 active+clean; 695 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:55 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:55 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:55 smithi131 ceph-mon[41654]: pgmap v1706: 176 pgs: 176 active+clean; 695 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:55 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:55 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:55.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:55 smithi131 ceph-mon[29507]: pgmap v1706: 176 pgs: 176 active+clean; 695 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:55.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:55 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:55.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:55 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:56.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:56 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:23:56.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:56 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:23:56.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:56 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:56.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:56 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:23:56.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:56 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:23:56.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:56 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:56.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:56 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:23:56.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:56 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:23:56.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:56 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:23:57.016 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 0 -ne 0 ']' 2024-02-05T08:23:57.016 INFO:tasks.workunit.client.0.smithi131.stderr:+ true 2024-02-05T08:23:57.016 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p b9122de8-c2e1-4670-8261-fe325aa3ccfb put twomore /etc/passwd 2024-02-05T08:23:57.054 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool set-quota b9122de8-c2e1-4670-8261-fe325aa3ccfb max_bytes 100 2024-02-05T08:23:57.143 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.140+0000 7fc70645b640 1 -- 172.21.15.131:0/3363326167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc700106530 msgr2=0x7fc70010e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:57.143 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.140+0000 7fc70645b640 1 --2- 172.21.15.131:0/3363326167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc700106530 0x7fc70010e100 secure :-1 s=READY pgs=3084 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009cd0 tx=0x7fc6f802c0f0 comp rx=0 tx=0).stop 2024-02-05T08:23:57.143 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.140+0000 7fc70645b640 1 -- 172.21.15.131:0/3363326167 shutdown_connections 2024-02-05T08:23:57.144 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.140+0000 7fc70645b640 1 --2- 172.21.15.131:0/3363326167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc700106530 0x7fc70010e100 unknown :-1 s=CLOSED pgs=3084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:57.144 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.140+0000 7fc70645b640 1 --2- 172.21.15.131:0/3363326167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc700101e10 0x7fc700105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:57.144 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.140+0000 7fc70645b640 1 --2- 172.21.15.131:0/3363326167 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc700101460 0x7fc700101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:57.144 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.140+0000 7fc70645b640 1 -- 172.21.15.131:0/3363326167 >> 172.21.15.131:0/3363326167 conn(0x7fc7000fcce0 msgr2=0x7fc7000ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:23:57.144 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.140+0000 7fc70645b640 1 -- 172.21.15.131:0/3363326167 shutdown_connections 2024-02-05T08:23:57.144 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.140+0000 7fc70645b640 1 -- 172.21.15.131:0/3363326167 wait complete. 2024-02-05T08:23:57.144 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.140+0000 7fc70645b640 1 Processor -- start 2024-02-05T08:23:57.144 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc70645b640 1 -- start start 2024-02-05T08:23:57.144 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc70645b640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc700101460 0x7fc700196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:57.144 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc70645b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc700106530 0x7fc700197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:57.144 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc70645b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70019b8b0 0x7fc70019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:57.144 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc70645b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70019c2a0 con 0x7fc70019b8b0 2024-02-05T08:23:57.145 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc70645b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70019c410 con 0x7fc700106530 2024-02-05T08:23:57.145 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc70645b640 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70019c580 con 0x7fc700101460 2024-02-05T08:23:57.145 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc6fffff640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc700101460 0x7fc700196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:57.145 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc7049f8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70019b8b0 0x7fc70019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:57.145 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc7049f8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70019b8b0 0x7fc70019bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53990/0 (socket says 172.21.15.131:53990) 2024-02-05T08:23:57.145 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc7049f8640 1 -- 172.21.15.131:0/3188039591 learned_addr learned my addr 172.21.15.131:0/3188039591 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T08:23:57.145 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc6fffff640 1 -- 172.21.15.131:0/3188039591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc700106530 msgr2=0x7fc700197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:57.145 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc6fffff640 1 --2- 172.21.15.131:0/3188039591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc700106530 0x7fc700197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:57.145 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc6fffff640 1 -- 172.21.15.131:0/3188039591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70019b8b0 msgr2=0x7fc70019bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:57.145 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc6fffff640 1 --2- 172.21.15.131:0/3188039591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70019b8b0 0x7fc70019bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:57.145 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc6fffff640 1 -- 172.21.15.131:0/3188039591 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f4009490 con 0x7fc700101460 2024-02-05T08:23:57.145 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc7049f8640 1 --2- 172.21.15.131:0/3188039591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70019b8b0 0x7fc70019bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:23:57.146 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc6fffff640 1 --2- 172.21.15.131:0/3188039591 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc700101460 0x7fc700196ff0 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400e9f0 tx=0x7fc6f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:23:57.146 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.141+0000 7fc6fd7fa640 1 -- 172.21.15.131:0/3188039591 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f400cd80 con 0x7fc700101460 2024-02-05T08:23:57.146 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.142+0000 7fc70645b640 1 -- 172.21.15.131:0/3188039591 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f8009900 con 0x7fc700101460 2024-02-05T08:23:57.146 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.142+0000 7fc6fd7fa640 1 -- 172.21.15.131:0/3188039591 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc6f40044d0 con 0x7fc700101460 2024-02-05T08:23:57.146 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.142+0000 7fc6fd7fa640 1 -- 172.21.15.131:0/3188039591 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4010430 con 0x7fc700101460 2024-02-05T08:23:57.146 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.142+0000 7fc70645b640 1 -- 172.21.15.131:0/3188039591 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7001a3e80 con 0x7fc700101460 2024-02-05T08:23:57.146 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.143+0000 7fc70645b640 1 -- 172.21.15.131:0/3188039591 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc700068560 con 0x7fc700101460 2024-02-05T08:23:57.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.143+0000 7fc6fd7fa640 1 -- 172.21.15.131:0/3188039591 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fc6f4003f70 con 0x7fc700101460 2024-02-05T08:23:57.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.143+0000 7fc6fd7fa640 1 --2- 172.21.15.131:0/3188039591 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc6e806d9f0 0x7fc6e806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:23:57.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.144+0000 7fc6ff7fe640 1 --2- 172.21.15.131:0/3188039591 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc6e806d9f0 0x7fc6e806feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:23:57.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.144+0000 7fc6fd7fa640 1 -- 172.21.15.131:0/3188039591 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(774..774 src has 254..774) v4 ==== 7814+0+0 (secure 0 0 0) 0x7fc6f4014070 con 0x7fc700101460 2024-02-05T08:23:57.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.144+0000 7fc6ff7fe640 1 --2- 172.21.15.131:0/3188039591 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc6e806d9f0 0x7fc6e806feb0 secure :-1 s=READY pgs=4078 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0005d80 tx=0x7fc6f0005c90 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:23:57.149 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.148+0000 7fc6fd7fa640 1 -- 172.21.15.131:0/3188039591 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc6f405acf0 con 0x7fc700101460 2024-02-05T08:23:57.323 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:57.322+0000 7fc70645b640 1 -- 172.21.15.131:0/3188039591 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7fc700198340 con 0x7fc700101460 2024-02-05T08:23:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:57 smithi157 ceph-mon[33345]: pgmap v1707: 176 pgs: 176 active+clean; 695 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:57 smithi157 ceph-mon[33345]: pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' no longer out of quota; removing NO_QUOTA flag 2024-02-05T08:23:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:57 smithi157 ceph-mon[33345]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:23:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:57 smithi157 ceph-mon[33345]: osdmap e774: 8 total, 8 up, 8 in 2024-02-05T08:23:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:57 smithi131 ceph-mon[41654]: pgmap v1707: 176 pgs: 176 active+clean; 695 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:57 smithi131 ceph-mon[41654]: pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' no longer out of quota; removing NO_QUOTA flag 2024-02-05T08:23:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:57 smithi131 ceph-mon[41654]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:23:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:57 smithi131 ceph-mon[41654]: osdmap e774: 8 total, 8 up, 8 in 2024-02-05T08:23:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:57 smithi131 ceph-mon[29507]: pgmap v1707: 176 pgs: 176 active+clean; 695 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:23:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:57 smithi131 ceph-mon[29507]: pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' no longer out of quota; removing NO_QUOTA flag 2024-02-05T08:23:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:57 smithi131 ceph-mon[29507]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:23:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:57 smithi131 ceph-mon[29507]: osdmap e774: 8 total, 8 up, 8 in 2024-02-05T08:23:58.072 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:58.071+0000 7fc6fd7fa640 1 -- 172.21.15.131:0/3188039591 <== mon.2 v2:172.21.15.131:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb v775) v1 ==== 221+0+0 (secure 0 0 0) 0x7fc700198340 con 0x7fc700101460 2024-02-05T08:23:58.180 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:58.179+0000 7fc70645b640 1 -- 172.21.15.131:0/3188039591 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7fc700198340 con 0x7fc700101460 2024-02-05T08:23:58.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:58 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3188039591' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:23:58.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:58 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:23:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:58 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3188039591' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:23:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:58 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:23:58.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:58 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3188039591' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:23:58.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:58 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:23:59.073 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:59.073+0000 7fc6fd7fa640 1 -- 172.21.15.131:0/3188039591 <== mon.2 v2:172.21.15.131:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb v776) v1 ==== 221+0+0 (secure 0 0 0) 0x7fc6f405a880 con 0x7fc700101460 2024-02-05T08:23:59.074 INFO:tasks.workunit.client.0.smithi131.stderr:set-quota max_bytes = 100 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb 2024-02-05T08:23:59.077 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:59.077+0000 7fc70645b640 1 -- 172.21.15.131:0/3188039591 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc6e806d9f0 msgr2=0x7fc6e806feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:59.077 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:59.077+0000 7fc70645b640 1 --2- 172.21.15.131:0/3188039591 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc6e806d9f0 0x7fc6e806feb0 secure :-1 s=READY pgs=4078 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0005d80 tx=0x7fc6f0005c90 comp rx=0 tx=0).stop 2024-02-05T08:23:59.077 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:59.077+0000 7fc70645b640 1 -- 172.21.15.131:0/3188039591 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc700101460 msgr2=0x7fc700196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:23:59.077 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:59.077+0000 7fc70645b640 1 --2- 172.21.15.131:0/3188039591 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc700101460 0x7fc700196ff0 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400e9f0 tx=0x7fc6f400eeb0 comp rx=0 tx=0).stop 2024-02-05T08:23:59.077 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:59.077+0000 7fc70645b640 1 -- 172.21.15.131:0/3188039591 shutdown_connections 2024-02-05T08:23:59.077 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:59.077+0000 7fc70645b640 1 --2- 172.21.15.131:0/3188039591 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc6e806d9f0 0x7fc6e806feb0 unknown :-1 s=CLOSED pgs=4078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:59.077 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:59.077+0000 7fc70645b640 1 --2- 172.21.15.131:0/3188039591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70019b8b0 0x7fc70019bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:59.077 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:59.077+0000 7fc70645b640 1 --2- 172.21.15.131:0/3188039591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc700106530 0x7fc700197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:59.077 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:59.077+0000 7fc70645b640 1 --2- 172.21.15.131:0/3188039591 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc700101460 0x7fc700196ff0 unknown :-1 s=CLOSED pgs=3089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:23:59.078 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:59.077+0000 7fc70645b640 1 -- 172.21.15.131:0/3188039591 >> 172.21.15.131:0/3188039591 conn(0x7fc7000fcce0 msgr2=0x7fc7000fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:23:59.078 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:59.077+0000 7fc70645b640 1 -- 172.21.15.131:0/3188039591 shutdown_connections 2024-02-05T08:23:59.078 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:23:59.077+0000 7fc70645b640 1 -- 172.21.15.131:0/3188039591 wait complete. 2024-02-05T08:23:59.089 INFO:tasks.workunit.client.0.smithi131.stderr:+ sleep 30 2024-02-05T08:23:59.371 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:59 smithi157 ceph-mon[33345]: pgmap v1709: 176 pgs: 176 active+clean; 695 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:59.372 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]': finished 2024-02-05T08:23:59.372 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:59 smithi157 ceph-mon[33345]: osdmap e775: 8 total, 8 up, 8 in 2024-02-05T08:23:59.372 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:59 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3188039591' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:23:59.372 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:23:59 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:23:59.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:59 smithi131 ceph-mon[41654]: pgmap v1709: 176 pgs: 176 active+clean; 695 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:59.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]': finished 2024-02-05T08:23:59.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:59 smithi131 ceph-mon[41654]: osdmap e775: 8 total, 8 up, 8 in 2024-02-05T08:23:59.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:59 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3188039591' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:23:59.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:23:59 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:23:59.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:59 smithi131 ceph-mon[29507]: pgmap v1709: 176 pgs: 176 active+clean; 695 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:23:59.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]': finished 2024-02-05T08:23:59.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:59 smithi131 ceph-mon[29507]: osdmap e775: 8 total, 8 up, 8 in 2024-02-05T08:23:59.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:59 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3188039591' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:23:59.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:23:59 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:24:00.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:24:00 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:24:00.386 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:00 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]': finished 2024-02-05T08:24:00.386 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:00 smithi157 ceph-mon[33345]: osdmap e776: 8 total, 8 up, 8 in 2024-02-05T08:24:00.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:00 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]': finished 2024-02-05T08:24:00.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:00 smithi131 ceph-mon[41654]: osdmap e776: 8 total, 8 up, 8 in 2024-02-05T08:24:00.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:00 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "100"}]': finished 2024-02-05T08:24:00.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:00 smithi131 ceph-mon[29507]: osdmap e776: 8 total, 8 up, 8 in 2024-02-05T08:24:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:01 smithi157 ceph-mon[33345]: pgmap v1712: 176 pgs: 176 active+clean; 717 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:24:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:01 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:01 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:24:01.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:01 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:24:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:01 smithi131 ceph-mon[41654]: pgmap v1712: 176 pgs: 176 active+clean; 717 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:24:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:01 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:01 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:24:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:01 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:24:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:01 smithi131 ceph-mon[29507]: pgmap v1712: 176 pgs: 176 active+clean; 717 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:24:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:01 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:01 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:24:01.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:01 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:24:02.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:02 smithi157 ceph-mon[33345]: pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' is full (reached quota's max_bytes: 100 B) 2024-02-05T08:24:02.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:02 smithi157 ceph-mon[33345]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:24:02.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:02 smithi157 ceph-mon[33345]: osdmap e777: 8 total, 8 up, 8 in 2024-02-05T08:24:02.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:02 smithi131 ceph-mon[41654]: pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' is full (reached quota's max_bytes: 100 B) 2024-02-05T08:24:02.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:02 smithi131 ceph-mon[41654]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:24:02.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:02 smithi131 ceph-mon[41654]: osdmap e777: 8 total, 8 up, 8 in 2024-02-05T08:24:02.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:02 smithi131 ceph-mon[29507]: pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' is full (reached quota's max_bytes: 100 B) 2024-02-05T08:24:02.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:02 smithi131 ceph-mon[29507]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:24:02.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:02 smithi131 ceph-mon[29507]: osdmap e777: 8 total, 8 up, 8 in 2024-02-05T08:24:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:03 smithi131 ceph-mon[41654]: pgmap v1713: 176 pgs: 176 active+clean; 717 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:24:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:03 smithi131 ceph-mon[29507]: pgmap v1713: 176 pgs: 176 active+clean; 717 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:24:03.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:24:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:24:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:24:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:03 smithi157 ceph-mon[33345]: pgmap v1713: 176 pgs: 176 active+clean; 717 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:24:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:05 smithi157 ceph-mon[33345]: pgmap v1715: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:24:05.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:05 smithi131 ceph-mon[41654]: pgmap v1715: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:24:05.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:05 smithi131 ceph-mon[29507]: pgmap v1715: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:24:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:07 smithi157 ceph-mon[33345]: pgmap v1716: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-05T08:24:07.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:07 smithi131 ceph-mon[41654]: pgmap v1716: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-05T08:24:07.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:07 smithi131 ceph-mon[29507]: pgmap v1716: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-05T08:24:09.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:09 smithi157 ceph-mon[33345]: pgmap v1717: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 573 B/s rd, 0 op/s 2024-02-05T08:24:09.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:09 smithi131 ceph-mon[41654]: pgmap v1717: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 573 B/s rd, 0 op/s 2024-02-05T08:24:09.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:09 smithi131 ceph-mon[29507]: pgmap v1717: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 573 B/s rd, 0 op/s 2024-02-05T08:24:10.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:24:10 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:24:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:11 smithi157 ceph-mon[33345]: pgmap v1718: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:24:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:11 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:11 smithi131 ceph-mon[41654]: pgmap v1718: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:24:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:11 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:11 smithi131 ceph-mon[29507]: pgmap v1718: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:24:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:11 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:13 smithi131 ceph-mon[41654]: pgmap v1719: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:24:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:13 smithi131 ceph-mon[29507]: pgmap v1719: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:24:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:24:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:24:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:24:13.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:13 smithi157 ceph-mon[33345]: pgmap v1719: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:24:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:15 smithi157 ceph-mon[33345]: pgmap v1720: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:24:15.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:15 smithi131 ceph-mon[41654]: pgmap v1720: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:24:15.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:15 smithi131 ceph-mon[29507]: pgmap v1720: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:24:17.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:16 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:24:17.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:24:17.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:16 smithi131 ceph-mon[41654]: pgmap v1721: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:16 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:24:17.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:24:17.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:16 smithi131 ceph-mon[29507]: pgmap v1721: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:16 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:24:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:24:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:16 smithi157 ceph-mon[33345]: pgmap v1721: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:19.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:19 smithi131 ceph-mon[41654]: pgmap v1722: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:19.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:19 smithi131 ceph-mon[29507]: pgmap v1722: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:19.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:19 smithi157 ceph-mon[33345]: pgmap v1722: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:20.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:24:20 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:24:21.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:21 smithi131 ceph-mon[41654]: pgmap v1723: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:24:21.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:21 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:21 smithi131 ceph-mon[29507]: pgmap v1723: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:24:21.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:21 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:21 smithi157 ceph-mon[33345]: pgmap v1723: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:24:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:21 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:23 smithi131 ceph-mon[41654]: pgmap v1724: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:23.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:23 smithi131 ceph-mon[29507]: pgmap v1724: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:24:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:24:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:24:23.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:23 smithi157 ceph-mon[33345]: pgmap v1724: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:25.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:25 smithi157 ceph-mon[33345]: pgmap v1725: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:24:25.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:25 smithi131 ceph-mon[29507]: pgmap v1725: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:24:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:25 smithi131 ceph-mon[41654]: pgmap v1725: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:24:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:27 smithi157 ceph-mon[33345]: pgmap v1726: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:27.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:27 smithi131 ceph-mon[41654]: pgmap v1726: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:27.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:27 smithi131 ceph-mon[29507]: pgmap v1726: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:29.091 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=159124 2024-02-05T08:24:29.092 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool set-quota b9122de8-c2e1-4670-8261-fe325aa3ccfb max_bytes 0 2024-02-05T08:24:29.092 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p b9122de8-c2e1-4670-8261-fe325aa3ccfb put two /etc/passwd 2024-02-05T08:24:29.188 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.186+0000 7f7454084640 1 -- 172.21.15.131:0/1788508656 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f744c105450 msgr2=0x7f744c10d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:29.189 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.186+0000 7f7454084640 1 --2- 172.21.15.131:0/1788508656 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f744c105450 0x7f744c10d020 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 crypto rx=0x7f7444009ca0 tx=0x7f744402c0f0 comp rx=0 tx=0).stop 2024-02-05T08:24:29.189 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.186+0000 7f7454084640 1 -- 172.21.15.131:0/1788508656 shutdown_connections 2024-02-05T08:24:29.189 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.186+0000 7f7454084640 1 --2- 172.21.15.131:0/1788508656 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f744c105450 0x7f744c10d020 unknown :-1 s=CLOSED pgs=3091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:29.189 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.186+0000 7f7454084640 1 --2- 172.21.15.131:0/1788508656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f744c100d30 0x7f744c104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:29.189 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.186+0000 7f7454084640 1 --2- 172.21.15.131:0/1788508656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f744c100380 0x7f744c100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:29.189 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.186+0000 7f7454084640 1 -- 172.21.15.131:0/1788508656 >> 172.21.15.131:0/1788508656 conn(0x7f744c077310 msgr2=0x7f744c077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:24:29.189 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.186+0000 7f7454084640 1 -- 172.21.15.131:0/1788508656 shutdown_connections 2024-02-05T08:24:29.189 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.186+0000 7f7454084640 1 -- 172.21.15.131:0/1788508656 wait complete. 2024-02-05T08:24:29.189 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7454084640 1 Processor -- start 2024-02-05T08:24:29.189 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7454084640 1 -- start start 2024-02-05T08:24:29.189 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7454084640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f744c100380 0x7f744c196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:29.189 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7454084640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f744c105450 0x7f744c1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:29.189 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7454084640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f744c19b870 0x7f744c19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:29.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7454084640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c19c260 con 0x7f744c100380 2024-02-05T08:24:29.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7454084640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c19c3d0 con 0x7f744c105450 2024-02-05T08:24:29.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7454084640 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744c19c540 con 0x7f744c19b870 2024-02-05T08:24:29.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7451e20640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f744c100380 0x7f744c196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:29.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7452621640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f744c19b870 0x7f744c19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:29.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7451e20640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f744c100380 0x7f744c196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38246/0 (socket says 172.21.15.131:38246) 2024-02-05T08:24:29.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7451e20640 1 -- 172.21.15.131:0/2096477900 learned_addr learned my addr 172.21.15.131:0/2096477900 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T08:24:29.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f745161f640 1 --2- 172.21.15.131:0/2096477900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f744c105450 0x7f744c1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:29.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7451e20640 1 -- 172.21.15.131:0/2096477900 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f744c19b870 msgr2=0x7f744c19bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:29.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7451e20640 1 --2- 172.21.15.131:0/2096477900 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f744c19b870 0x7f744c19bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:29.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7451e20640 1 -- 172.21.15.131:0/2096477900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f744c105450 msgr2=0x7f744c1974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:29.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7451e20640 1 --2- 172.21.15.131:0/2096477900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f744c105450 0x7f744c1974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:29.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f7451e20640 1 -- 172.21.15.131:0/2096477900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c009490 con 0x7f744c100380 2024-02-05T08:24:29.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.187+0000 7f745161f640 1 --2- 172.21.15.131:0/2096477900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f744c105450 0x7f744c1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T08:24:29.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.188+0000 7f7451e20640 1 --2- 172.21.15.131:0/2096477900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f744c100380 0x7f744c196fb0 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 crypto rx=0x7f743c00ea60 tx=0x7f743c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:24:29.191 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.188+0000 7f7442ffd640 1 -- 172.21.15.131:0/2096477900 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c00cde0 con 0x7f744c100380 2024-02-05T08:24:29.191 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.188+0000 7f7454084640 1 -- 172.21.15.131:0/2096477900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7444009900 con 0x7f744c100380 2024-02-05T08:24:29.191 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.188+0000 7f7442ffd640 1 -- 172.21.15.131:0/2096477900 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f743c0044d0 con 0x7f744c100380 2024-02-05T08:24:29.191 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.188+0000 7f7442ffd640 1 -- 172.21.15.131:0/2096477900 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c010690 con 0x7f744c100380 2024-02-05T08:24:29.191 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.188+0000 7f7454084640 1 -- 172.21.15.131:0/2096477900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744c1a3e90 con 0x7f744c100380 2024-02-05T08:24:29.191 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.189+0000 7f7454084640 1 -- 172.21.15.131:0/2096477900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744c068560 con 0x7f744c100380 2024-02-05T08:24:29.192 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.189+0000 7f7442ffd640 1 -- 172.21.15.131:0/2096477900 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f743c003e10 con 0x7f744c100380 2024-02-05T08:24:29.192 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.190+0000 7f7442ffd640 1 --2- 172.21.15.131:0/2096477900 >> v2:172.21.15.131:6800/2614238408 conn(0x7f743806d9f0 0x7f743806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:29.192 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.190+0000 7f7442ffd640 1 -- 172.21.15.131:0/2096477900 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(777..777 src has 254..777) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f743c014070 con 0x7f744c100380 2024-02-05T08:24:29.192 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.190+0000 7f745161f640 1 --2- 172.21.15.131:0/2096477900 >> v2:172.21.15.131:6800/2614238408 conn(0x7f743806d9f0 0x7f743806feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:29.192 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.190+0000 7f7442ffd640 1 -- 172.21.15.131:0/2096477900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=778}) v3 -- 0x7f7438078f30 con 0x7f744c100380 2024-02-05T08:24:29.192 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.190+0000 7f745161f640 1 --2- 172.21.15.131:0/2096477900 >> v2:172.21.15.131:6800/2614238408 conn(0x7f743806d9f0 0x7f743806feb0 secure :-1 s=READY pgs=4080 cs=0 l=1 rev1=1 crypto rx=0x7f7448005fd0 tx=0x7f744800b5d0 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:24:29.194 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.195+0000 7f7442ffd640 1 -- 172.21.15.131:0/2096477900 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f743c05ad30 con 0x7f744c100380 2024-02-05T08:24:29.370 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:29.370+0000 7f7454084640 1 -- 172.21.15.131:0/2096477900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f744c100d30 con 0x7f744c100380 2024-02-05T08:24:29.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:29 smithi157 ceph-mon[33345]: pgmap v1727: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:29 smithi131 ceph-mon[41654]: pgmap v1727: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:29 smithi131 ceph-mon[29507]: pgmap v1727: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:30.035 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:30.034+0000 7f7442ffd640 1 -- 172.21.15.131:0/2096477900 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb v778) v1 ==== 217+0+0 (secure 0 0 0) 0x7f743c05a8c0 con 0x7f744c100380 2024-02-05T08:24:30.035 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:30.035+0000 7f7442ffd640 1 -- 172.21.15.131:0/2096477900 <== mon.0 v2:172.21.15.131:3300/0 8 ==== osd_map(778..778 src has 254..778) v4 ==== 628+0+0 (secure 0 0 0) 0x7f743c003780 con 0x7f744c100380 2024-02-05T08:24:30.035 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:30.035+0000 7f7442ffd640 1 -- 172.21.15.131:0/2096477900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=779}) v3 -- 0x7f7438079e50 con 0x7f744c100380 2024-02-05T08:24:30.144 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:30.144+0000 7f7454084640 1 -- 172.21.15.131:0/2096477900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f744c100f40 con 0x7f744c100380 2024-02-05T08:24:30.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:30 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2096477900' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:24:30.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:24:30 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:24:30.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:30 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2096477900' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:24:30.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:30 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2096477900' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:24:31.036 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.035+0000 7f7442ffd640 1 -- 172.21.15.131:0/2096477900 <== mon.0 v2:172.21.15.131:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb v779) v1 ==== 217+0+0 (secure 0 0 0) 0x7f743c05aaa0 con 0x7f744c100380 2024-02-05T08:24:31.037 INFO:tasks.workunit.client.0.smithi131.stderr:set-quota max_bytes = 0 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb 2024-02-05T08:24:31.039 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.038+0000 7f7442ffd640 1 -- 172.21.15.131:0/2096477900 <== mon.0 v2:172.21.15.131:3300/0 10 ==== osd_map(779..779 src has 254..779) v4 ==== 628+0+0 (secure 0 0 0) 0x7f743c05b660 con 0x7f744c100380 2024-02-05T08:24:31.039 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.038+0000 7f7442ffd640 1 -- 172.21.15.131:0/2096477900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=780}) v3 -- 0x7f7438079e90 con 0x7f744c100380 2024-02-05T08:24:31.040 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.038+0000 7f7454084640 1 -- 172.21.15.131:0/2096477900 >> v2:172.21.15.131:6800/2614238408 conn(0x7f743806d9f0 msgr2=0x7f743806feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:31.040 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.038+0000 7f7454084640 1 --2- 172.21.15.131:0/2096477900 >> v2:172.21.15.131:6800/2614238408 conn(0x7f743806d9f0 0x7f743806feb0 secure :-1 s=READY pgs=4080 cs=0 l=1 rev1=1 crypto rx=0x7f7448005fd0 tx=0x7f744800b5d0 comp rx=0 tx=0).stop 2024-02-05T08:24:31.040 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.038+0000 7f7454084640 1 -- 172.21.15.131:0/2096477900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f744c100380 msgr2=0x7f744c196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:31.040 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.038+0000 7f7454084640 1 --2- 172.21.15.131:0/2096477900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f744c100380 0x7f744c196fb0 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 crypto rx=0x7f743c00ea60 tx=0x7f743c00ef20 comp rx=0 tx=0).stop 2024-02-05T08:24:31.040 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.038+0000 7f7454084640 1 -- 172.21.15.131:0/2096477900 shutdown_connections 2024-02-05T08:24:31.040 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.038+0000 7f7454084640 1 --2- 172.21.15.131:0/2096477900 >> v2:172.21.15.131:6800/2614238408 conn(0x7f743806d9f0 0x7f743806feb0 unknown :-1 s=CLOSED pgs=4080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:31.040 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.038+0000 7f7454084640 1 --2- 172.21.15.131:0/2096477900 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f744c19b870 0x7f744c19bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:31.040 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.039+0000 7f7454084640 1 --2- 172.21.15.131:0/2096477900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f744c105450 0x7f744c1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:31.040 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.039+0000 7f7454084640 1 --2- 172.21.15.131:0/2096477900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f744c100380 0x7f744c196fb0 unknown :-1 s=CLOSED pgs=3087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:31.040 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.039+0000 7f7454084640 1 -- 172.21.15.131:0/2096477900 >> 172.21.15.131:0/2096477900 conn(0x7f744c077310 msgr2=0x7f744c103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:24:31.040 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.039+0000 7f7454084640 1 -- 172.21.15.131:0/2096477900 shutdown_connections 2024-02-05T08:24:31.040 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.039+0000 7f7454084640 1 -- 172.21.15.131:0/2096477900 wait complete. 2024-02-05T08:24:31.056 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool set-quota b9122de8-c2e1-4670-8261-fe325aa3ccfb max_objects 0 2024-02-05T08:24:31.146 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.143+0000 7f349412b640 1 -- 172.21.15.131:0/1261779735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f348c105430 msgr2=0x7f348c10d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:31.146 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.143+0000 7f349412b640 1 --2- 172.21.15.131:0/1261779735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f348c105430 0x7f348c10d000 secure :-1 s=READY pgs=3088 cs=0 l=1 rev1=1 crypto rx=0x7f3484009cd0 tx=0x7f348402c0f0 comp rx=0 tx=0).stop 2024-02-05T08:24:31.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.143+0000 7f349412b640 1 -- 172.21.15.131:0/1261779735 shutdown_connections 2024-02-05T08:24:31.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.143+0000 7f349412b640 1 --2- 172.21.15.131:0/1261779735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f348c105430 0x7f348c10d000 unknown :-1 s=CLOSED pgs=3088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:31.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.143+0000 7f349412b640 1 --2- 172.21.15.131:0/1261779735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f348c100d10 0x7f348c104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:31.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.143+0000 7f349412b640 1 --2- 172.21.15.131:0/1261779735 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f348c100360 0x7f348c100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:31.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.143+0000 7f349412b640 1 -- 172.21.15.131:0/1261779735 >> 172.21.15.131:0/1261779735 conn(0x7f348c077250 msgr2=0x7f348c077660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:24:31.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.143+0000 7f349412b640 1 -- 172.21.15.131:0/1261779735 shutdown_connections 2024-02-05T08:24:31.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.143+0000 7f349412b640 1 -- 172.21.15.131:0/1261779735 wait complete. 2024-02-05T08:24:31.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f349412b640 1 Processor -- start 2024-02-05T08:24:31.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f349412b640 1 -- start start 2024-02-05T08:24:31.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f349412b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f348c100360 0x7f348c195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:31.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f349412b640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f348c105430 0x7f348c196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:31.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f349412b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f348c19a6d0 0x7f348c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:31.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f349412b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c19b0c0 con 0x7f348c100360 2024-02-05T08:24:31.147 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f349412b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c19b230 con 0x7f348c19a6d0 2024-02-05T08:24:31.148 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f349412b640 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c19b3a0 con 0x7f348c105430 2024-02-05T08:24:31.148 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f3491ec7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f348c100360 0x7f348c195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:31.148 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f34916c6640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f348c105430 0x7f348c196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:31.148 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f3491ec7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f348c100360 0x7f348c195e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38260/0 (socket says 172.21.15.131:38260) 2024-02-05T08:24:31.148 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f3491ec7640 1 -- 172.21.15.131:0/949199684 learned_addr learned my addr 172.21.15.131:0/949199684 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T08:24:31.148 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f34926c8640 1 --2- 172.21.15.131:0/949199684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f348c19a6d0 0x7f348c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:31.148 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f3491ec7640 1 -- 172.21.15.131:0/949199684 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f348c105430 msgr2=0x7f348c196350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:31.148 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f3491ec7640 1 --2- 172.21.15.131:0/949199684 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f348c105430 0x7f348c196350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:31.149 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f3491ec7640 1 -- 172.21.15.131:0/949199684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f348c19a6d0 msgr2=0x7f348c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:31.149 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f3491ec7640 1 --2- 172.21.15.131:0/949199684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f348c19a6d0 0x7f348c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:31.149 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f3491ec7640 1 -- 172.21.15.131:0/949199684 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3488009490 con 0x7f348c100360 2024-02-05T08:24:31.149 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f34916c6640 1 --2- 172.21.15.131:0/949199684 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f348c105430 0x7f348c196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:24:31.149 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.144+0000 7f3491ec7640 1 --2- 172.21.15.131:0/949199684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f348c100360 0x7f348c195e10 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 crypto rx=0x7f348800ea60 tx=0x7f348800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:24:31.149 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.145+0000 7f3482ffd640 1 -- 172.21.15.131:0/949199684 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348800cde0 con 0x7f348c100360 2024-02-05T08:24:31.149 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.145+0000 7f349412b640 1 -- 172.21.15.131:0/949199684 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3484009900 con 0x7f348c100360 2024-02-05T08:24:31.149 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.145+0000 7f3482ffd640 1 -- 172.21.15.131:0/949199684 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f34880044d0 con 0x7f348c100360 2024-02-05T08:24:31.150 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.145+0000 7f3482ffd640 1 -- 172.21.15.131:0/949199684 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488010690 con 0x7f348c100360 2024-02-05T08:24:31.150 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.145+0000 7f349412b640 1 -- 172.21.15.131:0/949199684 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348c1a2cf0 con 0x7f348c100360 2024-02-05T08:24:31.150 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.146+0000 7f349412b640 1 -- 172.21.15.131:0/949199684 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348c068110 con 0x7f348c100360 2024-02-05T08:24:31.151 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.146+0000 7f3482ffd640 1 -- 172.21.15.131:0/949199684 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f3488003e10 con 0x7f348c100360 2024-02-05T08:24:31.151 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.146+0000 7f3482ffd640 1 --2- 172.21.15.131:0/949199684 >> v2:172.21.15.131:6800/2614238408 conn(0x7f347806d9f0 0x7f347806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:31.151 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.146+0000 7f34916c6640 1 --2- 172.21.15.131:0/949199684 >> v2:172.21.15.131:6800/2614238408 conn(0x7f347806d9f0 0x7f347806feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:31.151 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.146+0000 7f3482ffd640 1 -- 172.21.15.131:0/949199684 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(779..779 src has 254..779) v4 ==== 7814+0+0 (secure 0 0 0) 0x7f3488014070 con 0x7f348c100360 2024-02-05T08:24:31.151 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.147+0000 7f3482ffd640 1 -- 172.21.15.131:0/949199684 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=780}) v3 -- 0x7f3478078f40 con 0x7f348c100360 2024-02-05T08:24:31.151 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.147+0000 7f34916c6640 1 --2- 172.21.15.131:0/949199684 >> v2:172.21.15.131:6800/2614238408 conn(0x7f347806d9f0 0x7f347806feb0 secure :-1 s=READY pgs=4081 cs=0 l=1 rev1=1 crypto rx=0x7f348c0ffcc0 tx=0x7f347c00b390 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:24:31.151 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.151+0000 7f3482ffd640 1 -- 172.21.15.131:0/949199684 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f348805ad30 con 0x7f348c100360 2024-02-05T08:24:31.327 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:31.326+0000 7f349412b640 1 -- 172.21.15.131:0/949199684 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f348c100d10 con 0x7f348c100360 2024-02-05T08:24:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:31 smithi157 ceph-mon[33345]: pgmap v1728: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:24:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:31 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2096477900' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:24:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:31 smithi157 ceph-mon[33345]: osdmap e778: 8 total, 8 up, 8 in 2024-02-05T08:24:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:31 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:31 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2096477900' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:24:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:31 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:24:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:31 smithi131 ceph-mon[41654]: pgmap v1728: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:24:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:31 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2096477900' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:24:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:31 smithi131 ceph-mon[41654]: osdmap e778: 8 total, 8 up, 8 in 2024-02-05T08:24:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:31 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:31 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2096477900' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:24:31.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:31 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:24:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:31 smithi131 ceph-mon[29507]: pgmap v1728: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:24:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:31 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2096477900' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:24:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:31 smithi131 ceph-mon[29507]: osdmap e778: 8 total, 8 up, 8 in 2024-02-05T08:24:31.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:31 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:31.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:31 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2096477900' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:24:31.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:31 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:24:32.013 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:32.012+0000 7f3482ffd640 1 -- 172.21.15.131:0/949199684 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb v780) v1 ==== 221+0+0 (secure 0 0 0) 0x7f348805a8c0 con 0x7f348c100360 2024-02-05T08:24:32.015 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:32.014+0000 7f3482ffd640 1 -- 172.21.15.131:0/949199684 <== mon.0 v2:172.21.15.131:3300/0 8 ==== osd_map(780..780 src has 254..780) v4 ==== 628+0+0 (secure 0 0 0) 0x7f3488003780 con 0x7f348c100360 2024-02-05T08:24:32.015 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:32.014+0000 7f3482ffd640 1 -- 172.21.15.131:0/949199684 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=781}) v3 -- 0x7f3478079e60 con 0x7f348c100360 2024-02-05T08:24:32.129 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:32.128+0000 7f349412b640 1 -- 172.21.15.131:0/949199684 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f348c197160 con 0x7f348c100360 2024-02-05T08:24:32.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:32 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2096477900' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:24:32.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:32 smithi157 ceph-mon[33345]: osdmap e779: 8 total, 8 up, 8 in 2024-02-05T08:24:32.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:32 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/949199684' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:24:32.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:32 smithi157 ceph-mon[33345]: pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' no longer out of quota; removing NO_QUOTA flag 2024-02-05T08:24:32.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:32 smithi157 ceph-mon[33345]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:24:32.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:32 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/949199684' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:24:32.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:32 smithi157 ceph-mon[33345]: osdmap e780: 8 total, 8 up, 8 in 2024-02-05T08:24:32.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:32 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2096477900' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:24:32.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:32 smithi131 ceph-mon[41654]: osdmap e779: 8 total, 8 up, 8 in 2024-02-05T08:24:32.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:32 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/949199684' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:24:32.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:32 smithi131 ceph-mon[41654]: pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' no longer out of quota; removing NO_QUOTA flag 2024-02-05T08:24:32.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:32 smithi131 ceph-mon[41654]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:24:32.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:32 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/949199684' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:24:32.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:32 smithi131 ceph-mon[41654]: osdmap e780: 8 total, 8 up, 8 in 2024-02-05T08:24:32.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:32 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2096477900' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:24:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:32 smithi131 ceph-mon[29507]: osdmap e779: 8 total, 8 up, 8 in 2024-02-05T08:24:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:32 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/949199684' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:24:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:32 smithi131 ceph-mon[29507]: pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' no longer out of quota; removing NO_QUOTA flag 2024-02-05T08:24:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:32 smithi131 ceph-mon[29507]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:24:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:32 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/949199684' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:24:32.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:32 smithi131 ceph-mon[29507]: osdmap e780: 8 total, 8 up, 8 in 2024-02-05T08:24:33.065 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.064+0000 7f3482ffd640 1 -- 172.21.15.131:0/949199684 <== mon.0 v2:172.21.15.131:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb v781) v1 ==== 221+0+0 (secure 0 0 0) 0x7f348805aaa0 con 0x7f348c100360 2024-02-05T08:24:33.066 INFO:tasks.workunit.client.0.smithi131.stderr:set-quota max_objects = 0 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb 2024-02-05T08:24:33.069 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.066+0000 7f3482ffd640 1 -- 172.21.15.131:0/949199684 <== mon.0 v2:172.21.15.131:3300/0 10 ==== osd_map(781..781 src has 254..781) v4 ==== 628+0+0 (secure 0 0 0) 0x7f348805f000 con 0x7f348c100360 2024-02-05T08:24:33.069 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.067+0000 7f349412b640 1 -- 172.21.15.131:0/949199684 >> v2:172.21.15.131:6800/2614238408 conn(0x7f347806d9f0 msgr2=0x7f347806feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:33.069 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.067+0000 7f349412b640 1 --2- 172.21.15.131:0/949199684 >> v2:172.21.15.131:6800/2614238408 conn(0x7f347806d9f0 0x7f347806feb0 secure :-1 s=READY pgs=4081 cs=0 l=1 rev1=1 crypto rx=0x7f348c0ffcc0 tx=0x7f347c00b390 comp rx=0 tx=0).stop 2024-02-05T08:24:33.069 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.067+0000 7f349412b640 1 -- 172.21.15.131:0/949199684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f348c100360 msgr2=0x7f348c195e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:33.069 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.067+0000 7f349412b640 1 --2- 172.21.15.131:0/949199684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f348c100360 0x7f348c195e10 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 crypto rx=0x7f348800ea60 tx=0x7f348800ef20 comp rx=0 tx=0).stop 2024-02-05T08:24:33.069 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.067+0000 7f349412b640 1 -- 172.21.15.131:0/949199684 shutdown_connections 2024-02-05T08:24:33.070 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.067+0000 7f349412b640 1 --2- 172.21.15.131:0/949199684 >> v2:172.21.15.131:6800/2614238408 conn(0x7f347806d9f0 0x7f347806feb0 unknown :-1 s=CLOSED pgs=4081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:33.070 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.067+0000 7f349412b640 1 --2- 172.21.15.131:0/949199684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f348c19a6d0 0x7f348c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:33.070 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.067+0000 7f349412b640 1 --2- 172.21.15.131:0/949199684 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f348c105430 0x7f348c196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:33.070 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.067+0000 7f349412b640 1 --2- 172.21.15.131:0/949199684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f348c100360 0x7f348c195e10 unknown :-1 s=CLOSED pgs=3089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:33.070 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.067+0000 7f349412b640 1 -- 172.21.15.131:0/949199684 >> 172.21.15.131:0/949199684 conn(0x7f348c077250 msgr2=0x7f348c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:24:33.070 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.067+0000 7f349412b640 1 -- 172.21.15.131:0/949199684 shutdown_connections 2024-02-05T08:24:33.070 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.067+0000 7f349412b640 1 -- 172.21.15.131:0/949199684 wait complete. 2024-02-05T08:24:33.083 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 159124 2024-02-05T08:24:33.083 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 0 -ne 0 ']' 2024-02-05T08:24:33.083 INFO:tasks.workunit.client.0.smithi131.stderr:+ true 2024-02-05T08:24:33.083 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p b9122de8-c2e1-4670-8261-fe325aa3ccfb put three /etc/passwd 2024-02-05T08:24:33.120 INFO:tasks.workunit.client.0.smithi131.stderr:++ uuidgen 2024-02-05T08:24:33.121 INFO:tasks.workunit.client.0.smithi131.stderr:+ pp=264c4d54-5197-4f45-8a91-42b2f8b06ac6 2024-02-05T08:24:33.121 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool create 264c4d54-5197-4f45-8a91-42b2f8b06ac6 12 2024-02-05T08:24:33.206 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.202+0000 7faede585640 1 -- 172.21.15.131:0/221095925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faed810b820 msgr2=0x7faed810dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:33.206 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.202+0000 7faede585640 1 --2- 172.21.15.131:0/221095925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faed810b820 0x7faed810dc90 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 crypto rx=0x7faed0009ca0 tx=0x7faed002c180 comp rx=0 tx=0).stop 2024-02-05T08:24:33.206 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.202+0000 7faede585640 1 -- 172.21.15.131:0/221095925 shutdown_connections 2024-02-05T08:24:33.206 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.202+0000 7faede585640 1 --2- 172.21.15.131:0/221095925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faed810b820 0x7faed810dc90 unknown :-1 s=CLOSED pgs=3091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:33.206 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.202+0000 7faede585640 1 --2- 172.21.15.131:0/221095925 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7faed8102d70 0x7faed810b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:33.206 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.202+0000 7faede585640 1 --2- 172.21.15.131:0/221095925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed8102450 0x7faed8102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:33.206 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.202+0000 7faede585640 1 -- 172.21.15.131:0/221095925 >> 172.21.15.131:0/221095925 conn(0x7faed80772b0 msgr2=0x7faed80776c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:24:33.206 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.202+0000 7faede585640 1 -- 172.21.15.131:0/221095925 shutdown_connections 2024-02-05T08:24:33.206 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.202+0000 7faede585640 1 -- 172.21.15.131:0/221095925 wait complete. 2024-02-05T08:24:33.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.203+0000 7faede585640 1 Processor -- start 2024-02-05T08:24:33.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.203+0000 7faede585640 1 -- start start 2024-02-05T08:24:33.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.203+0000 7faede585640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7faed8102450 0x7faed8196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:33.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.203+0000 7faede585640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faed8102d70 0x7faed81974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:33.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.203+0000 7faede585640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed810b820 0x7faed819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:33.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.203+0000 7faede585640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed819be00 con 0x7faed8102d70 2024-02-05T08:24:33.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.203+0000 7faede585640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed819bf70 con 0x7faed810b820 2024-02-05T08:24:33.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.203+0000 7faede585640 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed819c130 con 0x7faed8102450 2024-02-05T08:24:33.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.203+0000 7faed7fff640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7faed8102450 0x7faed8196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:33.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.203+0000 7faed77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faed8102d70 0x7faed81974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:33.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.203+0000 7faed7fff640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7faed8102450 0x7faed8196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:45972/0 (socket says 172.21.15.131:45972) 2024-02-05T08:24:33.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.203+0000 7faed7fff640 1 -- 172.21.15.131:0/3946357231 learned_addr learned my addr 172.21.15.131:0/3946357231 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T08:24:33.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.204+0000 7faedcb22640 1 --2- 172.21.15.131:0/3946357231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed810b820 0x7faed819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:33.208 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.204+0000 7faed7fff640 1 -- 172.21.15.131:0/3946357231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed810b820 msgr2=0x7faed819b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:33.208 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.204+0000 7faed7fff640 1 --2- 172.21.15.131:0/3946357231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed810b820 0x7faed819b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:33.208 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.204+0000 7faed7fff640 1 -- 172.21.15.131:0/3946357231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faed8102d70 msgr2=0x7faed81974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:33.208 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.204+0000 7faed7fff640 1 --2- 172.21.15.131:0/3946357231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faed8102d70 0x7faed81974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:33.208 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.204+0000 7faed7fff640 1 -- 172.21.15.131:0/3946357231 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faecc009490 con 0x7faed8102450 2024-02-05T08:24:33.208 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.204+0000 7faed77fe640 1 --2- 172.21.15.131:0/3946357231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faed8102d70 0x7faed81974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:24:33.208 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.204+0000 7faed7fff640 1 --2- 172.21.15.131:0/3946357231 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7faed8102450 0x7faed8196f90 secure :-1 s=READY pgs=3094 cs=0 l=1 rev1=1 crypto rx=0x7faec800c890 tx=0x7faec800cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:24:33.208 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.204+0000 7faed57fa640 1 -- 172.21.15.131:0/3946357231 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec8004220 con 0x7faed8102450 2024-02-05T08:24:33.208 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.204+0000 7faede585640 1 -- 172.21.15.131:0/3946357231 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed0009900 con 0x7faed8102450 2024-02-05T08:24:33.208 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.204+0000 7faed57fa640 1 -- 172.21.15.131:0/3946357231 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7faec8007e60 con 0x7faed8102450 2024-02-05T08:24:33.208 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.204+0000 7faed57fa640 1 -- 172.21.15.131:0/3946357231 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec8005690 con 0x7faed8102450 2024-02-05T08:24:33.209 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.204+0000 7faede585640 1 -- 172.21.15.131:0/3946357231 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed81a3ab0 con 0x7faed8102450 2024-02-05T08:24:33.209 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.205+0000 7faede585640 1 -- 172.21.15.131:0/3946357231 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed8068560 con 0x7faed8102450 2024-02-05T08:24:33.209 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.206+0000 7faed57fa640 1 -- 172.21.15.131:0/3946357231 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7faec80079b0 con 0x7faed8102450 2024-02-05T08:24:33.209 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.206+0000 7faed57fa640 1 --2- 172.21.15.131:0/3946357231 >> v2:172.21.15.131:6800/2614238408 conn(0x7faec006d9f0 0x7faec006feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:33.209 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.206+0000 7faed57fa640 1 -- 172.21.15.131:0/3946357231 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(781..781 src has 254..781) v4 ==== 7814+0+0 (secure 0 0 0) 0x7faec808dab0 con 0x7faed8102450 2024-02-05T08:24:33.210 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.206+0000 7faed77fe640 1 --2- 172.21.15.131:0/3946357231 >> v2:172.21.15.131:6800/2614238408 conn(0x7faec006d9f0 0x7faec006feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:33.210 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.206+0000 7faed77fe640 1 --2- 172.21.15.131:0/3946357231 >> v2:172.21.15.131:6800/2614238408 conn(0x7faec006d9f0 0x7faec006feb0 secure :-1 s=READY pgs=4083 cs=0 l=1 rev1=1 crypto rx=0x7faed8077a60 tx=0x7faecc005f90 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:24:33.212 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.211+0000 7faed57fa640 1 -- 172.21.15.131:0/3946357231 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faec805a970 con 0x7faed8102450 2024-02-05T08:24:33.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:33 smithi131 ceph-mon[41654]: pgmap v1731: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:24:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:33 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/949199684' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:24:33.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:33 smithi131 ceph-mon[29507]: pgmap v1731: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:24:33.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:33 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/949199684' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:24:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:24:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:24:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:24:33.390 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:33.389+0000 7faede585640 1 -- 172.21.15.131:0/3946357231 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12} v 0) v1 -- 0x7faed8106020 con 0x7faed8102450 2024-02-05T08:24:33.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:33 smithi157 ceph-mon[33345]: pgmap v1731: 176 pgs: 176 active+clean; 717 KiB data, 857 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:24:33.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:33 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/949199684' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:24:34.078 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.077+0000 7faed57fa640 1 -- 172.21.15.131:0/3946357231 <== mon.2 v2:172.21.15.131:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]=0 pool '264c4d54-5197-4f45-8a91-42b2f8b06ac6' created v782) v1 ==== 176+0+0 (secure 0 0 0) 0x7faec801d0c0 con 0x7faed8102450 2024-02-05T08:24:34.186 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.185+0000 7faede585640 1 -- 172.21.15.131:0/3946357231 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12} v 0) v1 -- 0x7faed8110a20 con 0x7faed8102450 2024-02-05T08:24:34.187 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.186+0000 7faed57fa640 1 -- 172.21.15.131:0/3946357231 <== mon.2 v2:172.21.15.131:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]=0 pool '264c4d54-5197-4f45-8a91-42b2f8b06ac6' already exists v782) v1 ==== 183+0+0 (secure 0 0 0) 0x7faec801d0c0 con 0x7faed8102450 2024-02-05T08:24:34.187 INFO:tasks.workunit.client.0.smithi131.stderr:pool '264c4d54-5197-4f45-8a91-42b2f8b06ac6' already exists 2024-02-05T08:24:34.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.188+0000 7faede585640 1 -- 172.21.15.131:0/3946357231 >> v2:172.21.15.131:6800/2614238408 conn(0x7faec006d9f0 msgr2=0x7faec006feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:34.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.188+0000 7faede585640 1 --2- 172.21.15.131:0/3946357231 >> v2:172.21.15.131:6800/2614238408 conn(0x7faec006d9f0 0x7faec006feb0 secure :-1 s=READY pgs=4083 cs=0 l=1 rev1=1 crypto rx=0x7faed8077a60 tx=0x7faecc005f90 comp rx=0 tx=0).stop 2024-02-05T08:24:34.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.188+0000 7faede585640 1 -- 172.21.15.131:0/3946357231 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7faed8102450 msgr2=0x7faed8196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:34.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.188+0000 7faede585640 1 --2- 172.21.15.131:0/3946357231 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7faed8102450 0x7faed8196f90 secure :-1 s=READY pgs=3094 cs=0 l=1 rev1=1 crypto rx=0x7faec800c890 tx=0x7faec800cd50 comp rx=0 tx=0).stop 2024-02-05T08:24:34.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.188+0000 7faede585640 1 -- 172.21.15.131:0/3946357231 shutdown_connections 2024-02-05T08:24:34.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.188+0000 7faede585640 1 --2- 172.21.15.131:0/3946357231 >> v2:172.21.15.131:6800/2614238408 conn(0x7faec006d9f0 0x7faec006feb0 unknown :-1 s=CLOSED pgs=4083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:34.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.188+0000 7faede585640 1 --2- 172.21.15.131:0/3946357231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed810b820 0x7faed819b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:34.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.188+0000 7faede585640 1 --2- 172.21.15.131:0/3946357231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faed8102d70 0x7faed81974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:34.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.188+0000 7faede585640 1 --2- 172.21.15.131:0/3946357231 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7faed8102450 0x7faed8196f90 unknown :-1 s=CLOSED pgs=3094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:34.190 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.188+0000 7faede585640 1 -- 172.21.15.131:0/3946357231 >> 172.21.15.131:0/3946357231 conn(0x7faed80772b0 msgr2=0x7faed81031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:24:34.191 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.188+0000 7faede585640 1 -- 172.21.15.131:0/3946357231 shutdown_connections 2024-02-05T08:24:34.191 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.188+0000 7faede585640 1 -- 172.21.15.131:0/3946357231 wait complete. 2024-02-05T08:24:34.199 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool application enable 264c4d54-5197-4f45-8a91-42b2f8b06ac6 rados 2024-02-05T08:24:34.283 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.280+0000 7fb1de313640 1 -- 172.21.15.131:0/3296826081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d81054f0 msgr2=0x7fb1d810d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:34.284 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.280+0000 7fb1de313640 1 --2- 172.21.15.131:0/3296826081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d81054f0 0x7fb1d810d030 secure :-1 s=READY pgs=3092 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009cd0 tx=0x7fb1d002c0f0 comp rx=0 tx=0).stop 2024-02-05T08:24:34.284 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.280+0000 7fb1de313640 1 -- 172.21.15.131:0/3296826081 shutdown_connections 2024-02-05T08:24:34.284 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.280+0000 7fb1de313640 1 --2- 172.21.15.131:0/3296826081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d81054f0 0x7fb1d810d030 unknown :-1 s=CLOSED pgs=3092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:34.284 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.280+0000 7fb1de313640 1 --2- 172.21.15.131:0/3296826081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d8101820 0x7fb1d8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:34.284 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.280+0000 7fb1de313640 1 --2- 172.21.15.131:0/3296826081 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fb1d8100e70 0x7fb1d8101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:34.284 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.280+0000 7fb1de313640 1 -- 172.21.15.131:0/3296826081 >> 172.21.15.131:0/3296826081 conn(0x7fb1d8075450 msgr2=0x7fb1d8075860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:24:34.284 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.280+0000 7fb1de313640 1 -- 172.21.15.131:0/3296826081 shutdown_connections 2024-02-05T08:24:34.284 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.280+0000 7fb1de313640 1 -- 172.21.15.131:0/3296826081 wait complete. 2024-02-05T08:24:34.284 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.280+0000 7fb1de313640 1 Processor -- start 2024-02-05T08:24:34.284 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.280+0000 7fb1de313640 1 -- start start 2024-02-05T08:24:34.284 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1de313640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fb1d8100e70 0x7fb1d8196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:34.284 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1de313640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d81054f0 0x7fb1d81974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:34.285 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1de313640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d819b840 0x7fb1d819bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:34.285 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1de313640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d819c230 con 0x7fb1d819b840 2024-02-05T08:24:34.285 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1de313640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d819c3a0 con 0x7fb1d81054f0 2024-02-05T08:24:34.285 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1de313640 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d819c510 con 0x7fb1d8100e70 2024-02-05T08:24:34.285 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1d7fff640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fb1d8100e70 0x7fb1d8196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:34.285 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1dc8b0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d819b840 0x7fb1d819bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:34.285 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1d7fff640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fb1d8100e70 0x7fb1d8196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:45992/0 (socket says 172.21.15.131:45992) 2024-02-05T08:24:34.285 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1dc8b0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d819b840 0x7fb1d819bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38330/0 (socket says 172.21.15.131:38330) 2024-02-05T08:24:34.285 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1dc8b0640 1 -- 172.21.15.131:0/3910382792 learned_addr learned my addr 172.21.15.131:0/3910382792 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T08:24:34.285 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1d77fe640 1 --2- 172.21.15.131:0/3910382792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d81054f0 0x7fb1d81974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:34.285 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1dc8b0640 1 -- 172.21.15.131:0/3910382792 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fb1d8100e70 msgr2=0x7fb1d8196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:34.285 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1dc8b0640 1 --2- 172.21.15.131:0/3910382792 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fb1d8100e70 0x7fb1d8196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:34.285 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1dc8b0640 1 -- 172.21.15.131:0/3910382792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d81054f0 msgr2=0x7fb1d81974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:34.285 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1dc8b0640 1 --2- 172.21.15.131:0/3910382792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d81054f0 0x7fb1d81974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:34.286 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1dc8b0640 1 -- 172.21.15.131:0/3910382792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d0009900 con 0x7fb1d819b840 2024-02-05T08:24:34.286 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1d7fff640 1 --2- 172.21.15.131:0/3910382792 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fb1d8100e70 0x7fb1d8196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:24:34.286 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1dc8b0640 1 --2- 172.21.15.131:0/3910382792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d819b840 0x7fb1d819bcf0 secure :-1 s=READY pgs=3093 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009ca0 tx=0x7fb1d00329a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:24:34.286 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.281+0000 7fb1d57fa640 1 -- 172.21.15.131:0/3910382792 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0034e70 con 0x7fb1d819b840 2024-02-05T08:24:34.286 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.282+0000 7fb1d57fa640 1 -- 172.21.15.131:0/3910382792 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fb1d0032e50 con 0x7fb1d819b840 2024-02-05T08:24:34.287 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.282+0000 7fb1de313640 1 -- 172.21.15.131:0/3910382792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d81a3af0 con 0x7fb1d819b840 2024-02-05T08:24:34.287 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.282+0000 7fb1d57fa640 1 -- 172.21.15.131:0/3910382792 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0004760 con 0x7fb1d819b840 2024-02-05T08:24:34.287 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.282+0000 7fb1de313640 1 -- 172.21.15.131:0/3910382792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d81a3f10 con 0x7fb1d819b840 2024-02-05T08:24:34.287 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.283+0000 7fb1de313640 1 -- 172.21.15.131:0/3910382792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d8068560 con 0x7fb1d819b840 2024-02-05T08:24:34.288 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.283+0000 7fb1d57fa640 1 -- 172.21.15.131:0/3910382792 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fb1d003b040 con 0x7fb1d819b840 2024-02-05T08:24:34.288 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.284+0000 7fb1d57fa640 1 --2- 172.21.15.131:0/3910382792 >> v2:172.21.15.131:6800/2614238408 conn(0x7fb1c006d9f0 0x7fb1c006feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:34.288 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.284+0000 7fb1d57fa640 1 -- 172.21.15.131:0/3910382792 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(782..782 src has 254..782) v4 ==== 8189+0+0 (secure 0 0 0) 0x7fb1d00b0120 con 0x7fb1d819b840 2024-02-05T08:24:34.288 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.284+0000 7fb1d7fff640 1 --2- 172.21.15.131:0/3910382792 >> v2:172.21.15.131:6800/2614238408 conn(0x7fb1c006d9f0 0x7fb1c006feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:34.288 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.284+0000 7fb1d7fff640 1 --2- 172.21.15.131:0/3910382792 >> v2:172.21.15.131:6800/2614238408 conn(0x7fb1c006d9f0 0x7fb1c006feb0 secure :-1 s=READY pgs=4084 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009fc0 tx=0x7fb1c8005f90 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:24:34.289 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.288+0000 7fb1d57fa640 1 -- 172.21.15.131:0/3910382792 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb1d007ce20 con 0x7fb1d819b840 2024-02-05T08:24:34.464 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:34.463+0000 7fb1de313640 1 -- 172.21.15.131:0/3910382792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "app": "rados"} v 0) v1 -- 0x7fb1d8101820 con 0x7fb1d819b840 2024-02-05T08:24:34.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/949199684' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:24:34.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:34 smithi157 ceph-mon[33345]: osdmap e781: 8 total, 8 up, 8 in 2024-02-05T08:24:34.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:34 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3946357231' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]: dispatch 2024-02-05T08:24:34.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:34 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]: dispatch 2024-02-05T08:24:34.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/949199684' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:24:34.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:34 smithi131 ceph-mon[41654]: osdmap e781: 8 total, 8 up, 8 in 2024-02-05T08:24:34.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:34 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3946357231' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]: dispatch 2024-02-05T08:24:34.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:34 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]: dispatch 2024-02-05T08:24:34.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/949199684' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:24:34.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:34 smithi131 ceph-mon[29507]: osdmap e781: 8 total, 8 up, 8 in 2024-02-05T08:24:34.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:34 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3946357231' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]: dispatch 2024-02-05T08:24:34.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:34 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]: dispatch 2024-02-05T08:24:35.075 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:35.074+0000 7fb1d57fa640 1 -- 172.21.15.131:0/3910382792 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "app": "rados"}]=0 enabled application 'rados' on pool '264c4d54-5197-4f45-8a91-42b2f8b06ac6' v783) v1 ==== 213+0+0 (secure 0 0 0) 0x7fb1d007c9b0 con 0x7fb1d819b840 2024-02-05T08:24:35.189 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:35.188+0000 7fb1de313640 1 -- 172.21.15.131:0/3910382792 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "app": "rados"} v 0) v1 -- 0x7fb1d810f970 con 0x7fb1d819b840 2024-02-05T08:24:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:35 smithi157 ceph-mon[33345]: pgmap v1734: 176 pgs: 176 active+clean; 740 KiB data, 861 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-05T08:24:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]': finished 2024-02-05T08:24:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:35 smithi157 ceph-mon[33345]: osdmap e782: 8 total, 8 up, 8 in 2024-02-05T08:24:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3946357231' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]: dispatch 2024-02-05T08:24:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:35 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]: dispatch 2024-02-05T08:24:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:35 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3910382792' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "app": "rados"}]: dispatch 2024-02-05T08:24:35.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:35 smithi131 ceph-mon[41654]: pgmap v1734: 176 pgs: 176 active+clean; 740 KiB data, 861 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-05T08:24:35.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]': finished 2024-02-05T08:24:35.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:35 smithi131 ceph-mon[41654]: osdmap e782: 8 total, 8 up, 8 in 2024-02-05T08:24:35.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3946357231' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]: dispatch 2024-02-05T08:24:35.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:35 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]: dispatch 2024-02-05T08:24:35.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:35 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3910382792' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "app": "rados"}]: dispatch 2024-02-05T08:24:35.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:35 smithi131 ceph-mon[29507]: pgmap v1734: 176 pgs: 176 active+clean; 740 KiB data, 861 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-05T08:24:35.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]': finished 2024-02-05T08:24:35.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:35 smithi131 ceph-mon[29507]: osdmap e782: 8 total, 8 up, 8 in 2024-02-05T08:24:35.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3946357231' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]: dispatch 2024-02-05T08:24:35.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:35 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pg_num": 12}]: dispatch 2024-02-05T08:24:35.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:35 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3910382792' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "app": "rados"}]: dispatch 2024-02-05T08:24:36.091 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.089+0000 7fb1d57fa640 1 -- 172.21.15.131:0/3910382792 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "app": "rados"}]=0 enabled application 'rados' on pool '264c4d54-5197-4f45-8a91-42b2f8b06ac6' v784) v1 ==== 213+0+0 (secure 0 0 0) 0x7fb1d00410d0 con 0x7fb1d819b840 2024-02-05T08:24:36.091 INFO:tasks.workunit.client.0.smithi131.stderr:enabled application 'rados' on pool '264c4d54-5197-4f45-8a91-42b2f8b06ac6' 2024-02-05T08:24:36.093 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.092+0000 7fb1de313640 1 -- 172.21.15.131:0/3910382792 >> v2:172.21.15.131:6800/2614238408 conn(0x7fb1c006d9f0 msgr2=0x7fb1c006feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:36.093 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.092+0000 7fb1de313640 1 --2- 172.21.15.131:0/3910382792 >> v2:172.21.15.131:6800/2614238408 conn(0x7fb1c006d9f0 0x7fb1c006feb0 secure :-1 s=READY pgs=4084 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009fc0 tx=0x7fb1c8005f90 comp rx=0 tx=0).stop 2024-02-05T08:24:36.093 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.092+0000 7fb1de313640 1 -- 172.21.15.131:0/3910382792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d819b840 msgr2=0x7fb1d819bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:36.093 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.092+0000 7fb1de313640 1 --2- 172.21.15.131:0/3910382792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d819b840 0x7fb1d819bcf0 secure :-1 s=READY pgs=3093 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009ca0 tx=0x7fb1d00329a0 comp rx=0 tx=0).stop 2024-02-05T08:24:36.094 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.092+0000 7fb1de313640 1 -- 172.21.15.131:0/3910382792 shutdown_connections 2024-02-05T08:24:36.094 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.092+0000 7fb1de313640 1 --2- 172.21.15.131:0/3910382792 >> v2:172.21.15.131:6800/2614238408 conn(0x7fb1c006d9f0 0x7fb1c006feb0 unknown :-1 s=CLOSED pgs=4084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:36.094 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.092+0000 7fb1de313640 1 --2- 172.21.15.131:0/3910382792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d819b840 0x7fb1d819bcf0 unknown :-1 s=CLOSED pgs=3093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:36.094 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.092+0000 7fb1de313640 1 --2- 172.21.15.131:0/3910382792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1d81054f0 0x7fb1d81974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:36.094 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.092+0000 7fb1de313640 1 --2- 172.21.15.131:0/3910382792 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fb1d8100e70 0x7fb1d8196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:36.094 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.092+0000 7fb1de313640 1 -- 172.21.15.131:0/3910382792 >> 172.21.15.131:0/3910382792 conn(0x7fb1d8075450 msgr2=0x7fb1d8103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:24:36.094 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.092+0000 7fb1de313640 1 -- 172.21.15.131:0/3910382792 shutdown_connections 2024-02-05T08:24:36.094 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.092+0000 7fb1de313640 1 -- 172.21.15.131:0/3910382792 wait complete. 2024-02-05T08:24:36.108 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool set-quota 264c4d54-5197-4f45-8a91-42b2f8b06ac6 max_objects 10 2024-02-05T08:24:36.197 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.194+0000 7fd3d595e640 1 -- 172.21.15.131:0/445003966 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fd3d0105450 msgr2=0x7fd3d010d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:36.197 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.194+0000 7fd3d595e640 1 --2- 172.21.15.131:0/445003966 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fd3d0105450 0x7fd3d010d020 secure :-1 s=READY pgs=3095 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009ca0 tx=0x7fd3c802c0f0 comp rx=0 tx=0).stop 2024-02-05T08:24:36.197 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.194+0000 7fd3d595e640 1 -- 172.21.15.131:0/445003966 shutdown_connections 2024-02-05T08:24:36.198 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.194+0000 7fd3d595e640 1 --2- 172.21.15.131:0/445003966 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fd3d0105450 0x7fd3d010d020 unknown :-1 s=CLOSED pgs=3095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:36.198 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.194+0000 7fd3d595e640 1 --2- 172.21.15.131:0/445003966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d0100d30 0x7fd3d0104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:36.198 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.194+0000 7fd3d595e640 1 --2- 172.21.15.131:0/445003966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3d0100380 0x7fd3d0100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:36.198 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.194+0000 7fd3d595e640 1 -- 172.21.15.131:0/445003966 >> 172.21.15.131:0/445003966 conn(0x7fd3d0077310 msgr2=0x7fd3d0077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:24:36.198 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.194+0000 7fd3d595e640 1 -- 172.21.15.131:0/445003966 shutdown_connections 2024-02-05T08:24:36.198 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.194+0000 7fd3d595e640 1 -- 172.21.15.131:0/445003966 wait complete. 2024-02-05T08:24:36.198 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.194+0000 7fd3d595e640 1 Processor -- start 2024-02-05T08:24:36.198 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.194+0000 7fd3d595e640 1 -- start start 2024-02-05T08:24:36.198 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3d595e640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fd3d0100380 0x7fd3d0197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:36.198 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3d595e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d0105450 0x7fd3d01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:36.198 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3d595e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3d0198880 0x7fd3d01a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:36.198 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3d595e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d01a3500 con 0x7fd3d0105450 2024-02-05T08:24:36.198 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3d595e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d01a3670 con 0x7fd3d0198880 2024-02-05T08:24:36.199 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3d595e640 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d01a3830 con 0x7fd3d0100380 2024-02-05T08:24:36.199 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3c67fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d0105450 0x7fd3d01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:36.199 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3ceffd640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fd3d0100380 0x7fd3d0197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:36.199 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3c67fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d0105450 0x7fd3d01982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38356/0 (socket says 172.21.15.131:38356) 2024-02-05T08:24:36.199 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3c67fc640 1 -- 172.21.15.131:0/3379850821 learned_addr learned my addr 172.21.15.131:0/3379850821 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T08:24:36.199 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3cf7fe640 1 --2- 172.21.15.131:0/3379850821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3d0198880 0x7fd3d01a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:36.199 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3ceffd640 1 -- 172.21.15.131:0/3379850821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3d0198880 msgr2=0x7fd3d01a2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:36.199 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3ceffd640 1 --2- 172.21.15.131:0/3379850821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3d0198880 0x7fd3d01a2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:36.199 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3ceffd640 1 -- 172.21.15.131:0/3379850821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d0105450 msgr2=0x7fd3d01982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:36.199 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3ceffd640 1 --2- 172.21.15.131:0/3379850821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d0105450 0x7fd3d01982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:36.199 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3ceffd640 1 -- 172.21.15.131:0/3379850821 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b8009490 con 0x7fd3d0100380 2024-02-05T08:24:36.199 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3c67fc640 1 --2- 172.21.15.131:0/3379850821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d0105450 0x7fd3d01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:24:36.200 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3ceffd640 1 --2- 172.21.15.131:0/3379850821 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fd3d0100380 0x7fd3d0197db0 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000c970 tx=0x7fd3c000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:24:36.200 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3d495c640 1 -- 172.21.15.131:0/3379850821 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0007bb0 con 0x7fd3d0100380 2024-02-05T08:24:36.200 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3d595e640 1 -- 172.21.15.131:0/3379850821 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c8009900 con 0x7fd3d0100380 2024-02-05T08:24:36.200 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3d495c640 1 -- 172.21.15.131:0/3379850821 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd3c0007d10 con 0x7fd3d0100380 2024-02-05T08:24:36.200 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3d495c640 1 -- 172.21.15.131:0/3379850821 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0004cf0 con 0x7fd3d0100380 2024-02-05T08:24:36.200 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.195+0000 7fd3d595e640 1 -- 172.21.15.131:0/3379850821 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d01a3ee0 con 0x7fd3d0100380 2024-02-05T08:24:36.200 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.196+0000 7fd3d595e640 1 -- 172.21.15.131:0/3379850821 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d0068560 con 0x7fd3d0100380 2024-02-05T08:24:36.204 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.197+0000 7fd3d495c640 1 -- 172.21.15.131:0/3379850821 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fd3c0004eb0 con 0x7fd3d0100380 2024-02-05T08:24:36.204 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.197+0000 7fd3d495c640 1 --2- 172.21.15.131:0/3379850821 >> v2:172.21.15.131:6800/2614238408 conn(0x7fd3bc06d9f0 0x7fd3bc06feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:24:36.204 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.197+0000 7fd3d495c640 1 -- 172.21.15.131:0/3379850821 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(784..784 src has 254..784) v4 ==== 8202+0+0 (secure 0 0 0) 0x7fd3c008d810 con 0x7fd3d0100380 2024-02-05T08:24:36.204 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.197+0000 7fd3c67fc640 1 --2- 172.21.15.131:0/3379850821 >> v2:172.21.15.131:6800/2614238408 conn(0x7fd3bc06d9f0 0x7fd3bc06feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:24:36.204 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.198+0000 7fd3c67fc640 1 --2- 172.21.15.131:0/3379850821 >> v2:172.21.15.131:6800/2614238408 conn(0x7fd3bc06d9f0 0x7fd3bc06feb0 secure :-1 s=READY pgs=4085 cs=0 l=1 rev1=1 crypto rx=0x7fd3d006a4a0 tx=0x7fd3b8005f90 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:24:36.204 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.202+0000 7fd3d495c640 1 -- 172.21.15.131:0/3379850821 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd3c005a500 con 0x7fd3d0100380 2024-02-05T08:24:36.376 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:36.375+0000 7fd3d595e640 1 -- 172.21.15.131:0/3379850821 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fd3d0100d30 con 0x7fd3d0100380 2024-02-05T08:24:36.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3910382792' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "app": "rados"}]': finished 2024-02-05T08:24:36.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:36 smithi157 ceph-mon[33345]: osdmap e783: 8 total, 8 up, 8 in 2024-02-05T08:24:36.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:36 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3910382792' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "app": "rados"}]: dispatch 2024-02-05T08:24:36.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3910382792' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "app": "rados"}]': finished 2024-02-05T08:24:36.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:36 smithi131 ceph-mon[41654]: osdmap e783: 8 total, 8 up, 8 in 2024-02-05T08:24:36.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:36 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3910382792' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "app": "rados"}]: dispatch 2024-02-05T08:24:36.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3910382792' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "app": "rados"}]': finished 2024-02-05T08:24:36.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:36 smithi131 ceph-mon[29507]: osdmap e783: 8 total, 8 up, 8 in 2024-02-05T08:24:36.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:36 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3910382792' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "app": "rados"}]: dispatch 2024-02-05T08:24:37.100 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:37.099+0000 7fd3d495c640 1 -- 172.21.15.131:0/3379850821 <== mon.2 v2:172.21.15.131:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 264c4d54-5197-4f45-8a91-42b2f8b06ac6 v785) v1 ==== 223+0+0 (secure 0 0 0) 0x7fd3c001d0d0 con 0x7fd3d0100380 2024-02-05T08:24:37.212 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:37.211+0000 7fd3d595e640 1 -- 172.21.15.131:0/3379850821 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fd3d0069f40 con 0x7fd3d0100380 2024-02-05T08:24:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:37 smithi157 ceph-mon[33345]: pgmap v1737: 188 pgs: 12 unknown, 176 active+clean; 740 KiB data, 861 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-05T08:24:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3910382792' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "app": "rados"}]': finished 2024-02-05T08:24:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:37 smithi157 ceph-mon[33345]: osdmap e784: 8 total, 8 up, 8 in 2024-02-05T08:24:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:37 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3379850821' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:24:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:37 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:24:37.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:37 smithi131 ceph-mon[41654]: pgmap v1737: 188 pgs: 12 unknown, 176 active+clean; 740 KiB data, 861 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-05T08:24:37.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3910382792' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "app": "rados"}]': finished 2024-02-05T08:24:37.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:37 smithi131 ceph-mon[41654]: osdmap e784: 8 total, 8 up, 8 in 2024-02-05T08:24:37.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:37 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3379850821' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:24:37.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:37 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:24:37.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:37 smithi131 ceph-mon[29507]: pgmap v1737: 188 pgs: 12 unknown, 176 active+clean; 740 KiB data, 861 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-05T08:24:37.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3910382792' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "app": "rados"}]': finished 2024-02-05T08:24:37.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:37 smithi131 ceph-mon[29507]: osdmap e784: 8 total, 8 up, 8 in 2024-02-05T08:24:37.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:37 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3379850821' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:24:37.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:37 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:24:38.119 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:38.118+0000 7fd3d495c640 1 -- 172.21.15.131:0/3379850821 <== mon.2 v2:172.21.15.131:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 264c4d54-5197-4f45-8a91-42b2f8b06ac6 v786) v1 ==== 223+0+0 (secure 0 0 0) 0x7fd3c005a090 con 0x7fd3d0100380 2024-02-05T08:24:38.119 INFO:tasks.workunit.client.0.smithi131.stderr:set-quota max_objects = 10 for pool 264c4d54-5197-4f45-8a91-42b2f8b06ac6 2024-02-05T08:24:38.121 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:38.120+0000 7fd3d595e640 1 -- 172.21.15.131:0/3379850821 >> v2:172.21.15.131:6800/2614238408 conn(0x7fd3bc06d9f0 msgr2=0x7fd3bc06feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:38.122 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:38.120+0000 7fd3d595e640 1 --2- 172.21.15.131:0/3379850821 >> v2:172.21.15.131:6800/2614238408 conn(0x7fd3bc06d9f0 0x7fd3bc06feb0 secure :-1 s=READY pgs=4085 cs=0 l=1 rev1=1 crypto rx=0x7fd3d006a4a0 tx=0x7fd3b8005f90 comp rx=0 tx=0).stop 2024-02-05T08:24:38.122 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:38.120+0000 7fd3d595e640 1 -- 172.21.15.131:0/3379850821 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fd3d0100380 msgr2=0x7fd3d0197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:24:38.122 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:38.120+0000 7fd3d595e640 1 --2- 172.21.15.131:0/3379850821 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fd3d0100380 0x7fd3d0197db0 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000c970 tx=0x7fd3c000ce30 comp rx=0 tx=0).stop 2024-02-05T08:24:38.122 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:38.121+0000 7fd3d595e640 1 -- 172.21.15.131:0/3379850821 shutdown_connections 2024-02-05T08:24:38.122 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:38.121+0000 7fd3d595e640 1 --2- 172.21.15.131:0/3379850821 >> v2:172.21.15.131:6800/2614238408 conn(0x7fd3bc06d9f0 0x7fd3bc06feb0 unknown :-1 s=CLOSED pgs=4085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:38.122 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:38.121+0000 7fd3d595e640 1 --2- 172.21.15.131:0/3379850821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3d0198880 0x7fd3d01a2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:38.122 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:38.121+0000 7fd3d595e640 1 --2- 172.21.15.131:0/3379850821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3d0105450 0x7fd3d01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:38.122 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:38.121+0000 7fd3d595e640 1 --2- 172.21.15.131:0/3379850821 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fd3d0100380 0x7fd3d0197db0 unknown :-1 s=CLOSED pgs=3096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:24:38.122 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:38.121+0000 7fd3d595e640 1 -- 172.21.15.131:0/3379850821 >> 172.21.15.131:0/3379850821 conn(0x7fd3d0077310 msgr2=0x7fd3d00fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:24:38.122 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:38.121+0000 7fd3d595e640 1 -- 172.21.15.131:0/3379850821 shutdown_connections 2024-02-05T08:24:38.122 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:24:38.121+0000 7fd3d595e640 1 -- 172.21.15.131:0/3379850821 wait complete. 2024-02-05T08:24:38.133 INFO:tasks.workunit.client.0.smithi131.stderr:+ sleep 30 2024-02-05T08:24:38.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:24:38.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:38 smithi157 ceph-mon[33345]: osdmap e785: 8 total, 8 up, 8 in 2024-02-05T08:24:38.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:38 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/3379850821' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:24:38.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:38 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:24:38.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:24:38.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:38 smithi131 ceph-mon[29507]: osdmap e785: 8 total, 8 up, 8 in 2024-02-05T08:24:38.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:38 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/3379850821' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:24:38.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:38 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:24:38.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:24:38.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:38 smithi131 ceph-mon[41654]: osdmap e785: 8 total, 8 up, 8 in 2024-02-05T08:24:38.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:38 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/3379850821' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:24:38.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:38 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:24:39.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:39 smithi157 ceph-mon[33345]: pgmap v1740: 188 pgs: 12 unknown, 176 active+clean; 740 KiB data, 861 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:24:39.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:24:39.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:39 smithi157 ceph-mon[33345]: osdmap e786: 8 total, 8 up, 8 in 2024-02-05T08:24:39.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:39 smithi131 ceph-mon[29507]: pgmap v1740: 188 pgs: 12 unknown, 176 active+clean; 740 KiB data, 861 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:24:39.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:24:39.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:39 smithi131 ceph-mon[29507]: osdmap e786: 8 total, 8 up, 8 in 2024-02-05T08:24:39.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:39 smithi131 ceph-mon[41654]: pgmap v1740: 188 pgs: 12 unknown, 176 active+clean; 740 KiB data, 861 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:24:39.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:24:39.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:39 smithi131 ceph-mon[41654]: osdmap e786: 8 total, 8 up, 8 in 2024-02-05T08:24:40.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:24:40 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:24:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:41 smithi157 ceph-mon[33345]: pgmap v1742: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:24:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:41 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:41 smithi131 ceph-mon[29507]: pgmap v1742: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:24:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:41 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:41 smithi131 ceph-mon[41654]: pgmap v1742: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:24:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:41 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:43.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:43 smithi131 ceph-mon[29507]: pgmap v1743: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:43 smithi131 ceph-mon[41654]: pgmap v1743: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:24:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:24:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:24:43.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:43 smithi157 ceph-mon[33345]: pgmap v1743: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:45.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:45 smithi157 ceph-mon[33345]: pgmap v1744: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:24:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:45 smithi131 ceph-mon[41654]: pgmap v1744: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:24:45.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:45 smithi131 ceph-mon[29507]: pgmap v1744: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:24:47.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:47 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:24:47.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:47 smithi131 ceph-mon[41654]: pgmap v1745: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:24:47.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:47 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:24:47.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:47 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:24:47.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:47 smithi131 ceph-mon[29507]: pgmap v1745: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:24:47.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:47 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:24:47.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:47 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:24:47.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:47 smithi157 ceph-mon[33345]: pgmap v1745: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T08:24:47.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:47 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:24:49.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:49 smithi157 ceph-mon[33345]: pgmap v1746: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:24:49.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:49 smithi131 ceph-mon[41654]: pgmap v1746: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:24:49.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:49 smithi131 ceph-mon[29507]: pgmap v1746: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:24:50.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:24:50 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:24:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:51 smithi157 ceph-mon[33345]: pgmap v1747: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:24:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:51 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:51.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:51 smithi131 ceph-mon[41654]: pgmap v1747: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:24:51.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:51 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:51 smithi131 ceph-mon[29507]: pgmap v1747: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T08:24:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:51 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:24:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:53 smithi131 ceph-mon[41654]: pgmap v1748: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:53.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:53 smithi131 ceph-mon[29507]: pgmap v1748: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:53.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:24:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:24:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:24:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:53 smithi157 ceph-mon[33345]: pgmap v1748: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:55 smithi157 ceph-mon[33345]: pgmap v1749: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:24:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:55 smithi131 ceph-mon[41654]: pgmap v1749: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:24:55.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:55 smithi131 ceph-mon[29507]: pgmap v1749: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:24:56.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:56 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:24:56.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:56 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:24:56.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:56 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:24:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:57 smithi157 ceph-mon[33345]: pgmap v1750: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:24:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:57 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:24:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:57 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:24:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:57 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:24:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:57 smithi131 ceph-mon[41654]: pgmap v1750: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:24:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:57 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:24:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:57 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:24:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:57 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:24:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:57 smithi131 ceph-mon[29507]: pgmap v1750: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:24:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:57 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:24:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:57 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:24:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:57 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:24:59.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:24:59 smithi157 ceph-mon[33345]: pgmap v1751: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:59.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:24:59 smithi131 ceph-mon[41654]: pgmap v1751: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:24:59.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:24:59 smithi131 ceph-mon[29507]: pgmap v1751: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:00.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:25:00 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:25:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:01 smithi157 ceph-mon[33345]: pgmap v1752: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:25:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:01 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:01 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:25:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:01 smithi131 ceph-mon[41654]: pgmap v1752: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:25:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:01 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:01.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:01 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:25:01.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:01 smithi131 ceph-mon[29507]: pgmap v1752: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:25:01.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:01 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:01.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:01 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:25:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:03 smithi131 ceph-mon[41654]: pgmap v1753: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:25:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:03 smithi131 ceph-mon[29507]: pgmap v1753: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:25:03.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:25:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:25:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:25:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:03 smithi157 ceph-mon[33345]: pgmap v1753: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:25:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:05 smithi157 ceph-mon[33345]: pgmap v1754: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:25:05.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:05 smithi131 ceph-mon[41654]: pgmap v1754: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:25:05.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:05 smithi131 ceph-mon[29507]: pgmap v1754: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:25:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:07 smithi157 ceph-mon[33345]: pgmap v1755: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:07.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:07 smithi131 ceph-mon[41654]: pgmap v1755: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:07.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:07 smithi131 ceph-mon[29507]: pgmap v1755: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:08.135 INFO:tasks.workunit.client.0.smithi131.stderr:++ seq 1 10 2024-02-05T08:25:08.136 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:25:08.136 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 264c4d54-5197-4f45-8a91-42b2f8b06ac6 put obj1 /etc/passwd 2024-02-05T08:25:08.177 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:25:08.177 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 264c4d54-5197-4f45-8a91-42b2f8b06ac6 put obj2 /etc/passwd 2024-02-05T08:25:08.214 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:25:08.214 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 264c4d54-5197-4f45-8a91-42b2f8b06ac6 put obj3 /etc/passwd 2024-02-05T08:25:08.251 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:25:08.251 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 264c4d54-5197-4f45-8a91-42b2f8b06ac6 put obj4 /etc/passwd 2024-02-05T08:25:08.286 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:25:08.286 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 264c4d54-5197-4f45-8a91-42b2f8b06ac6 put obj5 /etc/passwd 2024-02-05T08:25:08.321 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:25:08.321 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 264c4d54-5197-4f45-8a91-42b2f8b06ac6 put obj6 /etc/passwd 2024-02-05T08:25:08.355 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:25:08.355 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 264c4d54-5197-4f45-8a91-42b2f8b06ac6 put obj7 /etc/passwd 2024-02-05T08:25:08.389 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:25:08.389 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 264c4d54-5197-4f45-8a91-42b2f8b06ac6 put obj8 /etc/passwd 2024-02-05T08:25:08.424 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:25:08.424 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 264c4d54-5197-4f45-8a91-42b2f8b06ac6 put obj9 /etc/passwd 2024-02-05T08:25:08.458 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-02-05T08:25:08.458 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 264c4d54-5197-4f45-8a91-42b2f8b06ac6 put obj10 /etc/passwd 2024-02-05T08:25:08.493 INFO:tasks.workunit.client.0.smithi131.stderr:+ sleep 30 2024-02-05T08:25:09.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:09 smithi157 ceph-mon[33345]: pgmap v1756: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:09.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:09 smithi131 ceph-mon[41654]: pgmap v1756: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:09.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:09 smithi131 ceph-mon[29507]: pgmap v1756: 188 pgs: 188 active+clean; 740 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:10.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:25:10 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:25:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:11 smithi157 ceph-mon[33345]: pgmap v1757: 188 pgs: 188 active+clean; 785 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:25:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:11 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:11 smithi131 ceph-mon[41654]: pgmap v1757: 188 pgs: 188 active+clean; 785 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:25:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:11 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:11 smithi131 ceph-mon[29507]: pgmap v1757: 188 pgs: 188 active+clean; 785 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:25:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:11 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:13 smithi131 ceph-mon[41654]: pgmap v1758: 188 pgs: 188 active+clean; 785 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:25:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:13 smithi131 ceph-mon[29507]: pgmap v1758: 188 pgs: 188 active+clean; 785 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:25:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:25:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:25:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:25:13.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:13 smithi157 ceph-mon[33345]: pgmap v1758: 188 pgs: 188 active+clean; 785 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-05T08:25:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:15 smithi157 ceph-mon[33345]: pgmap v1759: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-05T08:25:15.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:15 smithi131 ceph-mon[41654]: pgmap v1759: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-05T08:25:15.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:15 smithi131 ceph-mon[29507]: pgmap v1759: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-05T08:25:16.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:16 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:25:16.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:16 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:25:16.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:16 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:25:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:17 smithi157 ceph-mon[33345]: pgmap v1760: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-05T08:25:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:17 smithi157 ceph-mon[33345]: pool '264c4d54-5197-4f45-8a91-42b2f8b06ac6' is full (reached quota's max_objects: 10) 2024-02-05T08:25:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:17 smithi157 ceph-mon[33345]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:25:17.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:17 smithi157 ceph-mon[33345]: osdmap e787: 8 total, 8 up, 8 in 2024-02-05T08:25:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:17 smithi131 ceph-mon[29507]: pgmap v1760: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-05T08:25:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:17 smithi131 ceph-mon[29507]: pool '264c4d54-5197-4f45-8a91-42b2f8b06ac6' is full (reached quota's max_objects: 10) 2024-02-05T08:25:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:17 smithi131 ceph-mon[29507]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:25:17.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:17 smithi131 ceph-mon[29507]: osdmap e787: 8 total, 8 up, 8 in 2024-02-05T08:25:17.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:17 smithi131 ceph-mon[41654]: pgmap v1760: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-05T08:25:17.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:17 smithi131 ceph-mon[41654]: pool '264c4d54-5197-4f45-8a91-42b2f8b06ac6' is full (reached quota's max_objects: 10) 2024-02-05T08:25:17.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:17 smithi131 ceph-mon[41654]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:25:17.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:17 smithi131 ceph-mon[41654]: osdmap e787: 8 total, 8 up, 8 in 2024-02-05T08:25:19.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:19 smithi157 ceph-mon[33345]: pgmap v1762: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-05T08:25:19.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:19 smithi131 ceph-mon[41654]: pgmap v1762: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-05T08:25:19.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:19 smithi131 ceph-mon[29507]: pgmap v1762: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-05T08:25:20.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:25:20 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:25:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:21 smithi157 ceph-mon[33345]: pgmap v1763: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 7.2 KiB/s wr, 1 op/s 2024-02-05T08:25:21.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:21 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:21 smithi131 ceph-mon[41654]: pgmap v1763: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 7.2 KiB/s wr, 1 op/s 2024-02-05T08:25:21.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:21 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:21 smithi131 ceph-mon[29507]: pgmap v1763: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 7.2 KiB/s wr, 1 op/s 2024-02-05T08:25:21.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:21 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:23.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:23 smithi131 ceph-mon[41654]: pgmap v1764: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 7.2 KiB/s wr, 1 op/s 2024-02-05T08:25:23.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:23 smithi131 ceph-mon[29507]: pgmap v1764: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 7.2 KiB/s wr, 1 op/s 2024-02-05T08:25:23.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:25:22 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:25:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:25:23.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:23 smithi157 ceph-mon[33345]: pgmap v1764: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 7.2 KiB/s wr, 1 op/s 2024-02-05T08:25:25.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:25 smithi157 ceph-mon[33345]: pgmap v1765: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:25:25.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:25 smithi131 ceph-mon[41654]: pgmap v1765: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:25:25.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:25 smithi131 ceph-mon[29507]: pgmap v1765: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:25:27.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:27 smithi157 ceph-mon[33345]: pgmap v1766: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:25:27.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:27 smithi131 ceph-mon[41654]: pgmap v1766: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:25:27.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:27 smithi131 ceph-mon[29507]: pgmap v1766: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:25:29.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:29 smithi157 ceph-mon[33345]: pgmap v1767: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 930 B/s rd, 0 op/s 2024-02-05T08:25:29.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:29 smithi131 ceph-mon[41654]: pgmap v1767: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 930 B/s rd, 0 op/s 2024-02-05T08:25:29.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:29 smithi131 ceph-mon[29507]: pgmap v1767: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 930 B/s rd, 0 op/s 2024-02-05T08:25:30.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:25:30 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:25:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:31 smithi157 ceph-mon[33345]: pgmap v1768: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:25:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:31 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:31 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:25:31.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:31 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:25:31.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:31 smithi131 ceph-mon[41654]: pgmap v1768: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:25:31.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:31 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:31.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:31 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:25:31.519 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:31 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:25:31.519 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:31 smithi131 ceph-mon[29507]: pgmap v1768: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:25:31.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:31 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:31.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:31 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:25:31.520 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:31 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:25:33.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:33 smithi131 ceph-mon[41654]: pgmap v1769: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:25:33.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:33 smithi131 ceph-mon[29507]: pgmap v1769: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:25:33.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:25:32 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:25:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:25:33.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:33 smithi157 ceph-mon[33345]: pgmap v1769: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:25:35.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:35 smithi157 ceph-mon[33345]: pgmap v1770: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:25:35.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:35 smithi131 ceph-mon[41654]: pgmap v1770: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:25:35.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:35 smithi131 ceph-mon[29507]: pgmap v1770: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:25:37.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:37 smithi157 ceph-mon[33345]: pgmap v1771: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:25:37.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:37 smithi131 ceph-mon[41654]: pgmap v1771: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:25:37.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:37 smithi131 ceph-mon[29507]: pgmap v1771: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:25:38.494 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p b9122de8-c2e1-4670-8261-fe325aa3ccfb put threemore /etc/passwd 2024-02-05T08:25:38.535 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool set-quota b9122de8-c2e1-4670-8261-fe325aa3ccfb max_bytes 0 2024-02-05T08:25:38.625 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.622+0000 7fc97ac9f640 1 -- 172.21.15.131:0/930634976 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc974100380 msgr2=0x7fc974100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:25:38.625 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.622+0000 7fc97ac9f640 1 --2- 172.21.15.131:0/930634976 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc974100380 0x7fc974100760 secure :-1 s=READY pgs=3112 cs=0 l=1 rev1=1 crypto rx=0x7fc964009900 tx=0x7fc96402bc30 comp rx=0 tx=0).stop 2024-02-05T08:25:38.625 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.623+0000 7fc97ac9f640 1 -- 172.21.15.131:0/930634976 shutdown_connections 2024-02-05T08:25:38.625 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.623+0000 7fc97ac9f640 1 --2- 172.21.15.131:0/930634976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc974105450 0x7fc97410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:38.625 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.623+0000 7fc97ac9f640 1 --2- 172.21.15.131:0/930634976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc974100d30 0x7fc974104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:38.625 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.623+0000 7fc97ac9f640 1 --2- 172.21.15.131:0/930634976 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc974100380 0x7fc974100760 unknown :-1 s=CLOSED pgs=3112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:38.625 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.623+0000 7fc97ac9f640 1 -- 172.21.15.131:0/930634976 >> 172.21.15.131:0/930634976 conn(0x7fc974077310 msgr2=0x7fc974077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:25:38.625 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.623+0000 7fc97ac9f640 1 -- 172.21.15.131:0/930634976 shutdown_connections 2024-02-05T08:25:38.626 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.623+0000 7fc97ac9f640 1 -- 172.21.15.131:0/930634976 wait complete. 2024-02-05T08:25:38.626 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.623+0000 7fc97ac9f640 1 Processor -- start 2024-02-05T08:25:38.626 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.623+0000 7fc97ac9f640 1 -- start start 2024-02-05T08:25:38.626 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.623+0000 7fc97ac9f640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc974100380 0x7fc974195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:25:38.626 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.623+0000 7fc97ac9f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc974105450 0x7fc9741963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:25:38.626 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.623+0000 7fc97ac9f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97419a730 0x7fc97419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:25:38.626 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.623+0000 7fc97ac9f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97419b120 con 0x7fc97419a730 2024-02-05T08:25:38.626 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.623+0000 7fc97ac9f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97419b290 con 0x7fc974105450 2024-02-05T08:25:38.626 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.623+0000 7fc97ac9f640 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97419b400 con 0x7fc974100380 2024-02-05T08:25:38.626 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc978a3b640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc974100380 0x7fc974195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:25:38.626 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc978a3b640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc974100380 0x7fc974195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:55552/0 (socket says 172.21.15.131:55552) 2024-02-05T08:25:38.626 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc97923c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97419a730 0x7fc97419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:25:38.626 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc978a3b640 1 -- 172.21.15.131:0/805813971 learned_addr learned my addr 172.21.15.131:0/805813971 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T08:25:38.626 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc973fff640 1 --2- 172.21.15.131:0/805813971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc974105450 0x7fc9741963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:25:38.626 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc978a3b640 1 -- 172.21.15.131:0/805813971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc974105450 msgr2=0x7fc9741963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:25:38.627 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc978a3b640 1 --2- 172.21.15.131:0/805813971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc974105450 0x7fc9741963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:38.627 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc978a3b640 1 -- 172.21.15.131:0/805813971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97419a730 msgr2=0x7fc97419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:25:38.627 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc978a3b640 1 --2- 172.21.15.131:0/805813971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97419a730 0x7fc97419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:38.627 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc978a3b640 1 -- 172.21.15.131:0/805813971 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc968009490 con 0x7fc974100380 2024-02-05T08:25:38.627 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc97923c640 1 --2- 172.21.15.131:0/805813971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97419a730 0x7fc97419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:25:38.627 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc978a3b640 1 --2- 172.21.15.131:0/805813971 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc974100380 0x7fc974195e70 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 crypto rx=0x7fc964000c00 tx=0x7fc964004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:25:38.627 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc971ffb640 1 -- 172.21.15.131:0/805813971 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964004500 con 0x7fc974100380 2024-02-05T08:25:38.627 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc971ffb640 1 -- 172.21.15.131:0/805813971 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc964003aa0 con 0x7fc974100380 2024-02-05T08:25:38.627 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc97ac9f640 1 -- 172.21.15.131:0/805813971 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc964009560 con 0x7fc974100380 2024-02-05T08:25:38.627 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc971ffb640 1 -- 172.21.15.131:0/805813971 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964005440 con 0x7fc974100380 2024-02-05T08:25:38.627 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.624+0000 7fc97ac9f640 1 -- 172.21.15.131:0/805813971 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc974069c20 con 0x7fc974100380 2024-02-05T08:25:38.628 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.625+0000 7fc97ac9f640 1 -- 172.21.15.131:0/805813971 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc974068560 con 0x7fc974100380 2024-02-05T08:25:38.628 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.626+0000 7fc971ffb640 1 -- 172.21.15.131:0/805813971 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fc9640055a0 con 0x7fc974100380 2024-02-05T08:25:38.628 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.626+0000 7fc971ffb640 1 --2- 172.21.15.131:0/805813971 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc95c06d9f0 0x7fc95c06feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:25:38.628 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.626+0000 7fc971ffb640 1 -- 172.21.15.131:0/805813971 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(787..787 src has 254..787) v4 ==== 8202+0+0 (secure 0 0 0) 0x7fc9640b2100 con 0x7fc974100380 2024-02-05T08:25:38.628 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.626+0000 7fc973fff640 1 --2- 172.21.15.131:0/805813971 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc95c06d9f0 0x7fc95c06feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:25:38.629 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.627+0000 7fc971ffb640 1 -- 172.21.15.131:0/805813971 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=788}) v3 -- 0x7fc95c079760 con 0x7fc974100380 2024-02-05T08:25:38.629 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.627+0000 7fc973fff640 1 --2- 172.21.15.131:0/805813971 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc95c06d9f0 0x7fc95c06feb0 secure :-1 s=READY pgs=4097 cs=0 l=1 rev1=1 crypto rx=0x7fc974104db0 tx=0x7fc968005f90 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:25:38.631 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.631+0000 7fc971ffb640 1 -- 172.21.15.131:0/805813971 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc96407edf0 con 0x7fc974100380 2024-02-05T08:25:38.805 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:38.805+0000 7fc97ac9f640 1 -- 172.21.15.131:0/805813971 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fc974100d30 con 0x7fc974100380 2024-02-05T08:25:39.044 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:39.044+0000 7fc971ffb640 1 -- 172.21.15.131:0/805813971 <== mon.2 v2:172.21.15.131:3301/0 7 ==== osd_map(788..788 src has 254..788) v4 ==== 628+0+0 (secure 0 0 0) 0x7fc9640830c0 con 0x7fc974100380 2024-02-05T08:25:39.047 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:39.047+0000 7fc971ffb640 1 -- 172.21.15.131:0/805813971 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=789}) v3 -- 0x7fc95c07a2d0 con 0x7fc974100380 2024-02-05T08:25:39.047 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:39.047+0000 7fc971ffb640 1 -- 172.21.15.131:0/805813971 <== mon.2 v2:172.21.15.131:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb v788) v1 ==== 217+0+0 (secure 0 0 0) 0x7fc96404d020 con 0x7fc974100380 2024-02-05T08:25:39.158 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:39.158+0000 7fc97ac9f640 1 -- 172.21.15.131:0/805813971 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fc974100f40 con 0x7fc974100380 2024-02-05T08:25:39.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:39 smithi157 ceph-mon[33345]: pgmap v1772: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:39.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:39 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/805813971' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:25:39.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:39 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:25:39.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:39 smithi131 ceph-mon[41654]: pgmap v1772: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:39.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:39 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/805813971' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:25:39.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:39 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:25:39.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:39 smithi131 ceph-mon[29507]: pgmap v1772: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:39.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:39 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/805813971' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:25:39.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:39 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:25:40.058 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.058+0000 7fc971ffb640 1 -- 172.21.15.131:0/805813971 <== mon.2 v2:172.21.15.131:3301/0 9 ==== osd_map(789..789 src has 254..789) v4 ==== 628+0+0 (secure 0 0 0) 0x7fc964087680 con 0x7fc974100380 2024-02-05T08:25:40.058 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.058+0000 7fc971ffb640 1 -- 172.21.15.131:0/805813971 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=790}) v3 -- 0x7fc95c07a740 con 0x7fc974100380 2024-02-05T08:25:40.063 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.063+0000 7fc971ffb640 1 -- 172.21.15.131:0/805813971 <== mon.2 v2:172.21.15.131:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb v789) v1 ==== 217+0+0 (secure 0 0 0) 0x7fc96404d030 con 0x7fc974100380 2024-02-05T08:25:40.063 INFO:tasks.workunit.client.0.smithi131.stderr:set-quota max_bytes = 0 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb 2024-02-05T08:25:40.067 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.067+0000 7fc97ac9f640 1 -- 172.21.15.131:0/805813971 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc95c06d9f0 msgr2=0x7fc95c06feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:25:40.067 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.067+0000 7fc97ac9f640 1 --2- 172.21.15.131:0/805813971 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc95c06d9f0 0x7fc95c06feb0 secure :-1 s=READY pgs=4097 cs=0 l=1 rev1=1 crypto rx=0x7fc974104db0 tx=0x7fc968005f90 comp rx=0 tx=0).stop 2024-02-05T08:25:40.067 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.067+0000 7fc97ac9f640 1 -- 172.21.15.131:0/805813971 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc974100380 msgr2=0x7fc974195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:25:40.067 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.067+0000 7fc97ac9f640 1 --2- 172.21.15.131:0/805813971 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc974100380 0x7fc974195e70 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 crypto rx=0x7fc964000c00 tx=0x7fc964004040 comp rx=0 tx=0).stop 2024-02-05T08:25:40.067 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.067+0000 7fc97ac9f640 1 -- 172.21.15.131:0/805813971 shutdown_connections 2024-02-05T08:25:40.067 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.067+0000 7fc97ac9f640 1 --2- 172.21.15.131:0/805813971 >> v2:172.21.15.131:6800/2614238408 conn(0x7fc95c06d9f0 0x7fc95c06feb0 unknown :-1 s=CLOSED pgs=4097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:40.067 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.067+0000 7fc97ac9f640 1 --2- 172.21.15.131:0/805813971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc97419a730 0x7fc97419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:40.067 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.067+0000 7fc97ac9f640 1 --2- 172.21.15.131:0/805813971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc974105450 0x7fc9741963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:40.067 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.067+0000 7fc97ac9f640 1 --2- 172.21.15.131:0/805813971 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fc974100380 0x7fc974195e70 unknown :-1 s=CLOSED pgs=3113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:40.067 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.067+0000 7fc97ac9f640 1 -- 172.21.15.131:0/805813971 >> 172.21.15.131:0/805813971 conn(0x7fc974077310 msgr2=0x7fc9740fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:25:40.068 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.067+0000 7fc97ac9f640 1 -- 172.21.15.131:0/805813971 shutdown_connections 2024-02-05T08:25:40.068 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.067+0000 7fc97ac9f640 1 -- 172.21.15.131:0/805813971 wait complete. 2024-02-05T08:25:40.078 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool set-quota b9122de8-c2e1-4670-8261-fe325aa3ccfb max_objects 0 2024-02-05T08:25:40.168 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.166+0000 7fe5d50ed640 1 -- 172.21.15.131:0/1244826319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5d0105440 msgr2=0x7fe5d010d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:25:40.168 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.166+0000 7fe5d50ed640 1 --2- 172.21.15.131:0/1244826319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5d0105440 0x7fe5d010d010 secure :-1 s=READY pgs=3112 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009c10 tx=0x7fe5c802bef0 comp rx=0 tx=0).stop 2024-02-05T08:25:40.168 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.166+0000 7fe5d50ed640 1 -- 172.21.15.131:0/1244826319 shutdown_connections 2024-02-05T08:25:40.168 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.166+0000 7fe5d50ed640 1 --2- 172.21.15.131:0/1244826319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5d0105440 0x7fe5d010d010 unknown :-1 s=CLOSED pgs=3112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:40.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.166+0000 7fe5d50ed640 1 --2- 172.21.15.131:0/1244826319 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe5d0100d20 0x7fe5d0104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:40.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.166+0000 7fe5d50ed640 1 --2- 172.21.15.131:0/1244826319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d0100370 0x7fe5d0100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:40.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.166+0000 7fe5d50ed640 1 -- 172.21.15.131:0/1244826319 >> 172.21.15.131:0/1244826319 conn(0x7fe5d00772b0 msgr2=0x7fe5d00776c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:25:40.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.166+0000 7fe5d50ed640 1 -- 172.21.15.131:0/1244826319 shutdown_connections 2024-02-05T08:25:40.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.166+0000 7fe5d50ed640 1 -- 172.21.15.131:0/1244826319 wait complete. 2024-02-05T08:25:40.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5d50ed640 1 Processor -- start 2024-02-05T08:25:40.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5d50ed640 1 -- start start 2024-02-05T08:25:40.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5d50ed640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe5d0100370 0x7fe5d010d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:25:40.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5d50ed640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5d0105440 0x7fe5d010de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:25:40.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5d50ed640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d010e370 0x7fe5d01a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:25:40.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5d50ed640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d01a5750 con 0x7fe5d0105440 2024-02-05T08:25:40.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5d50ed640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d01a58c0 con 0x7fe5d010e370 2024-02-05T08:25:40.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5d50ed640 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d01a5a80 con 0x7fe5d0100370 2024-02-05T08:25:40.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5ced9d640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe5d0100370 0x7fe5d010d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:25:40.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5c7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5d0105440 0x7fe5d010de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:25:40.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5ced9d640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe5d0100370 0x7fe5d010d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:55574/0 (socket says 172.21.15.131:55574) 2024-02-05T08:25:40.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5ced9d640 1 -- 172.21.15.131:0/2863973461 learned_addr learned my addr 172.21.15.131:0/2863973461 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T08:25:40.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5cf59e640 1 --2- 172.21.15.131:0/2863973461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d010e370 0x7fe5d01a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:25:40.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5ced9d640 1 -- 172.21.15.131:0/2863973461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d010e370 msgr2=0x7fe5d01a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:25:40.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5ced9d640 1 --2- 172.21.15.131:0/2863973461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d010e370 0x7fe5d01a5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:40.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5ced9d640 1 -- 172.21.15.131:0/2863973461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5d0105440 msgr2=0x7fe5d010de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:25:40.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5ced9d640 1 --2- 172.21.15.131:0/2863973461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5d0105440 0x7fe5d010de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:40.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5ced9d640 1 -- 172.21.15.131:0/2863973461 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b8009560 con 0x7fe5d0100370 2024-02-05T08:25:40.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.167+0000 7fe5c7fff640 1 --2- 172.21.15.131:0/2863973461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5d0105440 0x7fe5d010de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:25:40.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.168+0000 7fe5ced9d640 1 --2- 172.21.15.131:0/2863973461 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe5d0100370 0x7fe5d010d8f0 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000c920 tx=0x7fe5c000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:25:40.171 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.168+0000 7fe5ccd99640 1 -- 172.21.15.131:0/2863973461 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0007c70 con 0x7fe5d0100370 2024-02-05T08:25:40.171 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.168+0000 7fe5ccd99640 1 -- 172.21.15.131:0/2863973461 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fe5c0007dd0 con 0x7fe5d0100370 2024-02-05T08:25:40.171 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.168+0000 7fe5d50ed640 1 -- 172.21.15.131:0/2863973461 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c8009840 con 0x7fe5d0100370 2024-02-05T08:25:40.171 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.168+0000 7fe5ccd99640 1 -- 172.21.15.131:0/2863973461 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0018720 con 0x7fe5d0100370 2024-02-05T08:25:40.171 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.168+0000 7fe5d50ed640 1 -- 172.21.15.131:0/2863973461 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d01a60e0 con 0x7fe5d0100370 2024-02-05T08:25:40.172 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.169+0000 7fe5d50ed640 1 -- 172.21.15.131:0/2863973461 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d0068560 con 0x7fe5d0100370 2024-02-05T08:25:40.172 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.169+0000 7fe5ccd99640 1 -- 172.21.15.131:0/2863973461 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fe5c000f450 con 0x7fe5d0100370 2024-02-05T08:25:40.172 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.170+0000 7fe5ccd99640 1 --2- 172.21.15.131:0/2863973461 >> v2:172.21.15.131:6800/2614238408 conn(0x7fe5bc06da20 0x7fe5bc06fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:25:40.172 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.170+0000 7fe5c7fff640 1 --2- 172.21.15.131:0/2863973461 >> v2:172.21.15.131:6800/2614238408 conn(0x7fe5bc06da20 0x7fe5bc06fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:25:40.173 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.170+0000 7fe5ccd99640 1 -- 172.21.15.131:0/2863973461 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(789..789 src has 254..789) v4 ==== 8202+0+0 (secure 0 0 0) 0x7fe5c008dcc0 con 0x7fe5d0100370 2024-02-05T08:25:40.173 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.170+0000 7fe5ccd99640 1 -- 172.21.15.131:0/2863973461 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=790}) v3 -- 0x7fe5bc079790 con 0x7fe5d0100370 2024-02-05T08:25:40.173 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.170+0000 7fe5c7fff640 1 --2- 172.21.15.131:0/2863973461 >> v2:172.21.15.131:6800/2614238408 conn(0x7fe5bc06da20 0x7fe5bc06fee0 secure :-1 s=READY pgs=4098 cs=0 l=1 rev1=1 crypto rx=0x7fe5d010f380 tx=0x7fe5b8009260 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:25:40.174 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.175+0000 7fe5ccd99640 1 -- 172.21.15.131:0/2863973461 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe5c005a9b0 con 0x7fe5d0100370 2024-02-05T08:25:40.348 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:40.348+0000 7fe5d50ed640 1 -- 172.21.15.131:0/2863973461 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fe5d01086d0 con 0x7fe5d0100370 2024-02-05T08:25:40.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:25:40 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:25:40.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:25:40.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:40 smithi157 ceph-mon[33345]: osdmap e788: 8 total, 8 up, 8 in 2024-02-05T08:25:40.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:40 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/805813971' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:25:40.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:40 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:25:40.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:25:40.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:40 smithi131 ceph-mon[41654]: osdmap e788: 8 total, 8 up, 8 in 2024-02-05T08:25:40.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:40 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/805813971' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:25:40.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:40 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:25:40.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:25:40.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:40 smithi131 ceph-mon[29507]: osdmap e788: 8 total, 8 up, 8 in 2024-02-05T08:25:40.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:40 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/805813971' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:25:40.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:40 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:25:41.062 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:41.062+0000 7fe5ccd99640 1 -- 172.21.15.131:0/2863973461 <== mon.2 v2:172.21.15.131:3301/0 7 ==== osd_map(790..790 src has 254..790) v4 ==== 628+0+0 (secure 0 0 0) 0x7fe5c005ec80 con 0x7fe5d0100370 2024-02-05T08:25:41.063 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:41.062+0000 7fe5ccd99640 1 -- 172.21.15.131:0/2863973461 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=791}) v3 -- 0x7fe5bc07a2c0 con 0x7fe5d0100370 2024-02-05T08:25:41.066 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:41.066+0000 7fe5ccd99640 1 -- 172.21.15.131:0/2863973461 <== mon.2 v2:172.21.15.131:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb v790) v1 ==== 221+0+0 (secure 0 0 0) 0x7fe5c005be60 con 0x7fe5d0100370 2024-02-05T08:25:41.174 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:41.174+0000 7fe5d50ed640 1 -- 172.21.15.131:0/2863973461 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fe5d0100d20 con 0x7fe5d0100370 2024-02-05T08:25:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:41 smithi157 ceph-mon[33345]: pgmap v1774: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:25:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:25:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:41 smithi157 ceph-mon[33345]: osdmap e789: 8 total, 8 up, 8 in 2024-02-05T08:25:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:41 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:41.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:41 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2863973461' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:25:41.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:41 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:25:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:41 smithi131 ceph-mon[41654]: pgmap v1774: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:25:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:25:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:41 smithi131 ceph-mon[41654]: osdmap e789: 8 total, 8 up, 8 in 2024-02-05T08:25:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:41 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:41 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2863973461' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:25:41.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:41 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:25:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:41 smithi131 ceph-mon[29507]: pgmap v1774: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:25:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:25:41.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:41 smithi131 ceph-mon[29507]: osdmap e789: 8 total, 8 up, 8 in 2024-02-05T08:25:41.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:41 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:41.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:41 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2863973461' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:25:41.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:41 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:25:42.075 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:42.074+0000 7fe5ccd99640 1 -- 172.21.15.131:0/2863973461 <== mon.2 v2:172.21.15.131:3301/0 9 ==== osd_map(791..791 src has 254..791) v4 ==== 628+0+0 (secure 0 0 0) 0x7fe5c0063240 con 0x7fe5d0100370 2024-02-05T08:25:42.075 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:42.074+0000 7fe5ccd99640 1 -- 172.21.15.131:0/2863973461 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=792}) v3 -- 0x7fe5bc07a8c0 con 0x7fe5d0100370 2024-02-05T08:25:42.079 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:42.079+0000 7fe5ccd99640 1 -- 172.21.15.131:0/2863973461 <== mon.2 v2:172.21.15.131:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb v791) v1 ==== 221+0+0 (secure 0 0 0) 0x7fe5c005be60 con 0x7fe5d0100370 2024-02-05T08:25:42.079 INFO:tasks.workunit.client.0.smithi131.stderr:set-quota max_objects = 0 for pool b9122de8-c2e1-4670-8261-fe325aa3ccfb 2024-02-05T08:25:42.083 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:42.082+0000 7fe5d50ed640 1 -- 172.21.15.131:0/2863973461 >> v2:172.21.15.131:6800/2614238408 conn(0x7fe5bc06da20 msgr2=0x7fe5bc06fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:25:42.083 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:42.083+0000 7fe5d50ed640 1 --2- 172.21.15.131:0/2863973461 >> v2:172.21.15.131:6800/2614238408 conn(0x7fe5bc06da20 0x7fe5bc06fee0 secure :-1 s=READY pgs=4098 cs=0 l=1 rev1=1 crypto rx=0x7fe5d010f380 tx=0x7fe5b8009260 comp rx=0 tx=0).stop 2024-02-05T08:25:42.084 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:42.083+0000 7fe5d50ed640 1 -- 172.21.15.131:0/2863973461 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe5d0100370 msgr2=0x7fe5d010d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:25:42.084 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:42.083+0000 7fe5d50ed640 1 --2- 172.21.15.131:0/2863973461 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe5d0100370 0x7fe5d010d8f0 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000c920 tx=0x7fe5c000cde0 comp rx=0 tx=0).stop 2024-02-05T08:25:42.084 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:42.083+0000 7fe5d50ed640 1 -- 172.21.15.131:0/2863973461 shutdown_connections 2024-02-05T08:25:42.084 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:42.083+0000 7fe5d50ed640 1 --2- 172.21.15.131:0/2863973461 >> v2:172.21.15.131:6800/2614238408 conn(0x7fe5bc06da20 0x7fe5bc06fee0 unknown :-1 s=CLOSED pgs=4098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:42.084 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:42.083+0000 7fe5d50ed640 1 --2- 172.21.15.131:0/2863973461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5d010e370 0x7fe5d01a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:42.084 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:42.083+0000 7fe5d50ed640 1 --2- 172.21.15.131:0/2863973461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5d0105440 0x7fe5d010de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:42.085 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:42.083+0000 7fe5d50ed640 1 --2- 172.21.15.131:0/2863973461 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7fe5d0100370 0x7fe5d010d8f0 unknown :-1 s=CLOSED pgs=3114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:25:42.085 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:42.083+0000 7fe5d50ed640 1 -- 172.21.15.131:0/2863973461 >> 172.21.15.131:0/2863973461 conn(0x7fe5d00772b0 msgr2=0x7fe5d00fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:25:42.085 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:42.083+0000 7fe5d50ed640 1 -- 172.21.15.131:0/2863973461 shutdown_connections 2024-02-05T08:25:42.085 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:25:42.083+0000 7fe5d50ed640 1 -- 172.21.15.131:0/2863973461 wait complete. 2024-02-05T08:25:42.096 INFO:tasks.workunit.client.0.smithi131.stderr:+ sleep 30 2024-02-05T08:25:42.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:25:42.585 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:42 smithi157 ceph-mon[33345]: osdmap e790: 8 total, 8 up, 8 in 2024-02-05T08:25:42.585 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:42 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2863973461' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:25:42.585 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:42 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:25:42.586 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:25:42.586 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:42 smithi131 ceph-mon[41654]: osdmap e790: 8 total, 8 up, 8 in 2024-02-05T08:25:42.586 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:42 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2863973461' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:25:42.586 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:42 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:25:42.587 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:25:42.587 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:42 smithi131 ceph-mon[29507]: osdmap e790: 8 total, 8 up, 8 in 2024-02-05T08:25:42.587 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:42 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2863973461' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:25:42.587 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:42 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:25:43.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:25:42 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:25:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:25:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:43 smithi131 ceph-mon[41654]: pgmap v1777: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:43.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:43 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:25:43.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:43 smithi131 ceph-mon[41654]: osdmap e791: 8 total, 8 up, 8 in 2024-02-05T08:25:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:43 smithi131 ceph-mon[29507]: pgmap v1777: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:43 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:25:43.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:43 smithi131 ceph-mon[29507]: osdmap e791: 8 total, 8 up, 8 in 2024-02-05T08:25:43.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:43 smithi157 ceph-mon[33345]: pgmap v1777: 188 pgs: 188 active+clean; 852 KiB data, 862 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:43.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:43 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:25:43.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:43 smithi157 ceph-mon[33345]: osdmap e791: 8 total, 8 up, 8 in 2024-02-05T08:25:45.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:45 smithi157 ceph-mon[33345]: pgmap v1779: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-05T08:25:45.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:45 smithi131 ceph-mon[41654]: pgmap v1779: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-05T08:25:45.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:45 smithi131 ceph-mon[29507]: pgmap v1779: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-05T08:25:46.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:46 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:25:46.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:46 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:25:46.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:46 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:25:47.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:47 smithi157 ceph-mon[33345]: pgmap v1780: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-05T08:25:47.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:47 smithi131 ceph-mon[41654]: pgmap v1780: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-05T08:25:47.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:47 smithi131 ceph-mon[29507]: pgmap v1780: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-05T08:25:49.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:49 smithi157 ceph-mon[33345]: pgmap v1781: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 642 B/s rd, 1.5 KiB/s wr, 0 op/s 2024-02-05T08:25:49.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:49 smithi131 ceph-mon[41654]: pgmap v1781: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 642 B/s rd, 1.5 KiB/s wr, 0 op/s 2024-02-05T08:25:49.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:49 smithi131 ceph-mon[29507]: pgmap v1781: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 642 B/s rd, 1.5 KiB/s wr, 0 op/s 2024-02-05T08:25:50.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:25:50 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:25:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:51 smithi157 ceph-mon[33345]: pgmap v1782: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-02-05T08:25:51.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:51 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:51.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:51 smithi131 ceph-mon[41654]: pgmap v1782: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-02-05T08:25:51.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:51 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:51.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:51 smithi131 ceph-mon[29507]: pgmap v1782: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-02-05T08:25:51.518 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:51 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:25:53.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:53 smithi131 ceph-mon[41654]: pgmap v1783: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-05T08:25:53.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:25:52 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:25:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:25:53.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:53 smithi131 ceph-mon[29507]: pgmap v1783: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-05T08:25:53.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:53 smithi157 ceph-mon[33345]: pgmap v1783: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-05T08:25:55.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:55 smithi157 ceph-mon[33345]: pgmap v1784: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-02-05T08:25:55.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:55 smithi131 ceph-mon[29507]: pgmap v1784: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-02-05T08:25:55.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:55 smithi131 ceph-mon[41654]: pgmap v1784: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-02-05T08:25:57.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:57 smithi157 ceph-mon[33345]: pgmap v1785: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:57.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:57 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:25:57.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:57 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:25:57.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:57 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:25:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:57 smithi131 ceph-mon[29507]: pgmap v1785: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:57 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:25:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:57 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:25:57.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:57 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:25:57.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:57 smithi131 ceph-mon[41654]: pgmap v1785: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:57.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:57 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:25:57.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:57 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:25:57.518 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:57 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:25:58.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:58 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:25:58.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:58 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:25:58.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:58 smithi157 ceph-mon[33345]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:25:58.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:58 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:25:58.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:58 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:25:58.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:58 smithi131 ceph-mon[29507]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:25:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:58 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:25:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:58 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:25:58.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:58 smithi131 ceph-mon[41654]: from='mgr.14571 ' entity='mgr.y' 2024-02-05T08:25:59.371 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:25:59 smithi157 ceph-mon[33345]: pgmap v1786: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:59.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:25:59 smithi131 ceph-mon[41654]: pgmap v1786: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:25:59.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:25:59 smithi131 ceph-mon[29507]: pgmap v1786: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:26:00.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:26:00 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:26:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:01 smithi157 ceph-mon[33345]: pgmap v1787: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:26:01.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:01 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:26:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:01 smithi131 ceph-mon[41654]: pgmap v1787: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:26:01.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:01 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:26:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:01 smithi131 ceph-mon[29507]: pgmap v1787: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:26:01.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:01 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:26:02.472 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:02 smithi157 ceph-mon[33345]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:26:02.516 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:02 smithi131 ceph-mon[41654]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:26:02.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:02 smithi131 ceph-mon[29507]: from='mgr.14571 172.21.15.131:0/469218586' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:26:03.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:03 smithi131 ceph-mon[29507]: pgmap v1788: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:26:03.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:03 smithi131 ceph-mon[41654]: pgmap v1788: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:26:03.268 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:26:02 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:26:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:26:03.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:03 smithi157 ceph-mon[33345]: pgmap v1788: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:26:05.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:05 smithi157 ceph-mon[33345]: pgmap v1789: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:26:05.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:05 smithi131 ceph-mon[29507]: pgmap v1789: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:26:05.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:05 smithi131 ceph-mon[41654]: pgmap v1789: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:26:07.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:07 smithi157 ceph-mon[33345]: pgmap v1790: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:26:07.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:07 smithi131 ceph-mon[29507]: pgmap v1790: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:26:07.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:07 smithi131 ceph-mon[41654]: pgmap v1790: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:26:09.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:09 smithi157 ceph-mon[33345]: pgmap v1791: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:26:09.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:09 smithi131 ceph-mon[41654]: pgmap v1791: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:26:09.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:09 smithi131 ceph-mon[29507]: pgmap v1791: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:26:10.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:26:10 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug there is no tcmu-runner data available 2024-02-05T08:26:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:11 smithi157 ceph-mon[33345]: pgmap v1792: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:26:11.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:11 smithi157 ceph-mon[33345]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:26:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:11 smithi131 ceph-mon[41654]: pgmap v1792: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:26:11.517 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:11 smithi131 ceph-mon[41654]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:26:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:11 smithi131 ceph-mon[29507]: pgmap v1792: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:26:11.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:11 smithi131 ceph-mon[29507]: from='client.14532 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:26:12.098 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool delete b9122de8-c2e1-4670-8261-fe325aa3ccfb b9122de8-c2e1-4670-8261-fe325aa3ccfb --yes-i-really-really-mean-it 2024-02-05T08:26:12.193 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.190+0000 7f07406ae640 1 -- 172.21.15.131:0/2125667801 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f0738106540 msgr2=0x7f073810e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:26:12.193 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.190+0000 7f07406ae640 1 --2- 172.21.15.131:0/2125667801 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f0738106540 0x7f073810e110 secure :-1 s=READY pgs=3115 cs=0 l=1 rev1=1 crypto rx=0x7f0730009ca0 tx=0x7f073002c0f0 comp rx=0 tx=0).stop 2024-02-05T08:26:12.193 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.190+0000 7f07406ae640 1 -- 172.21.15.131:0/2125667801 shutdown_connections 2024-02-05T08:26:12.193 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.190+0000 7f07406ae640 1 --2- 172.21.15.131:0/2125667801 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f0738106540 0x7f073810e110 unknown :-1 s=CLOSED pgs=3115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:12.193 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.190+0000 7f07406ae640 1 --2- 172.21.15.131:0/2125667801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0738101e20 0x7f0738105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:12.193 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.190+0000 7f07406ae640 1 --2- 172.21.15.131:0/2125667801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0738101470 0x7f0738101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:12.193 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.190+0000 7f07406ae640 1 -- 172.21.15.131:0/2125667801 >> 172.21.15.131:0/2125667801 conn(0x7f07380fcd30 msgr2=0x7f07380ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:26:12.193 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.190+0000 7f07406ae640 1 -- 172.21.15.131:0/2125667801 shutdown_connections 2024-02-05T08:26:12.193 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.190+0000 7f07406ae640 1 -- 172.21.15.131:0/2125667801 wait complete. 2024-02-05T08:26:12.193 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.191+0000 7f07406ae640 1 Processor -- start 2024-02-05T08:26:12.193 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.191+0000 7f07406ae640 1 -- start start 2024-02-05T08:26:12.193 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.191+0000 7f07406ae640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f0738101470 0x7f0738196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:26:12.194 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.191+0000 7f07406ae640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0738106540 0x7f0738197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:26:12.194 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.191+0000 7f07406ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073819b7e0 0x7f073819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:26:12.194 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.191+0000 7f07406ae640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073819c1d0 con 0x7f0738106540 2024-02-05T08:26:12.194 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.191+0000 7f07406ae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073819c340 con 0x7f073819b7e0 2024-02-05T08:26:12.194 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.191+0000 7f07406ae640 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073819c4b0 con 0x7f0738101470 2024-02-05T08:26:12.194 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.191+0000 7f073e44a640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f0738101470 0x7f0738196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:26:12.194 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.191+0000 7f073e44a640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f0738101470 0x7f0738196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3301/0 says I am v2:172.21.15.131:44302/0 (socket says 172.21.15.131:44302) 2024-02-05T08:26:12.194 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.191+0000 7f073e44a640 1 -- 172.21.15.131:0/2931262618 learned_addr learned my addr 172.21.15.131:0/2931262618 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T08:26:12.194 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.191+0000 7f073dc49640 1 --2- 172.21.15.131:0/2931262618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0738106540 0x7f0738197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:26:12.194 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.192+0000 7f073ec4b640 1 --2- 172.21.15.131:0/2931262618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073819b7e0 0x7f073819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:26:12.194 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.192+0000 7f073e44a640 1 -- 172.21.15.131:0/2931262618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073819b7e0 msgr2=0x7f073819bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:26:12.194 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.192+0000 7f073e44a640 1 --2- 172.21.15.131:0/2931262618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073819b7e0 0x7f073819bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:12.194 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.192+0000 7f073e44a640 1 -- 172.21.15.131:0/2931262618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0738106540 msgr2=0x7f0738197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:26:12.194 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.192+0000 7f073e44a640 1 --2- 172.21.15.131:0/2931262618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0738106540 0x7f0738197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:12.194 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.192+0000 7f073e44a640 1 -- 172.21.15.131:0/2931262618 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0734009490 con 0x7f0738101470 2024-02-05T08:26:12.195 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.192+0000 7f073ec4b640 1 --2- 172.21.15.131:0/2931262618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073819b7e0 0x7f073819bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-05T08:26:12.195 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.192+0000 7f073dc49640 1 --2- 172.21.15.131:0/2931262618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0738106540 0x7f0738197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:26:12.195 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.192+0000 7f073e44a640 1 --2- 172.21.15.131:0/2931262618 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f0738101470 0x7f0738196f20 secure :-1 s=READY pgs=3116 cs=0 l=1 rev1=1 crypto rx=0x7f073400ea60 tx=0x7f073400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:26:12.195 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.192+0000 7f072f7fe640 1 -- 172.21.15.131:0/2931262618 <== mon.2 v2:172.21.15.131:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073400cde0 con 0x7f0738101470 2024-02-05T08:26:12.195 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.192+0000 7f072f7fe640 1 -- 172.21.15.131:0/2931262618 <== mon.2 v2:172.21.15.131:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f07340044d0 con 0x7f0738101470 2024-02-05T08:26:12.195 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.192+0000 7f072f7fe640 1 -- 172.21.15.131:0/2931262618 <== mon.2 v2:172.21.15.131:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0734010690 con 0x7f0738101470 2024-02-05T08:26:12.195 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.192+0000 7f07406ae640 1 -- 172.21.15.131:0/2931262618 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0730009900 con 0x7f0738101470 2024-02-05T08:26:12.195 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.192+0000 7f07406ae640 1 -- 172.21.15.131:0/2931262618 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07381a3e00 con 0x7f0738101470 2024-02-05T08:26:12.196 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.193+0000 7f07406ae640 1 -- 172.21.15.131:0/2931262618 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0738068560 con 0x7f0738101470 2024-02-05T08:26:12.196 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.193+0000 7f072f7fe640 1 -- 172.21.15.131:0/2931262618 <== mon.2 v2:172.21.15.131:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f0734003e10 con 0x7f0738101470 2024-02-05T08:26:12.196 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.194+0000 7f072f7fe640 1 --2- 172.21.15.131:0/2931262618 >> v2:172.21.15.131:6800/2614238408 conn(0x7f072406d9f0 0x7f072406feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:26:12.196 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.194+0000 7f073dc49640 1 --2- 172.21.15.131:0/2931262618 >> v2:172.21.15.131:6800/2614238408 conn(0x7f072406d9f0 0x7f072406feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:26:12.196 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.194+0000 7f072f7fe640 1 -- 172.21.15.131:0/2931262618 <== mon.2 v2:172.21.15.131:3301/0 5 ==== osd_map(791..791 src has 254..791) v4 ==== 8202+0+0 (secure 0 0 0) 0x7f0734014070 con 0x7f0738101470 2024-02-05T08:26:12.196 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.194+0000 7f072f7fe640 1 -- 172.21.15.131:0/2931262618 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=792}) v3 -- 0x7f07240797e0 con 0x7f0738101470 2024-02-05T08:26:12.196 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.194+0000 7f073dc49640 1 --2- 172.21.15.131:0/2931262618 >> v2:172.21.15.131:6800/2614238408 conn(0x7f072406d9f0 0x7f072406feb0 secure :-1 s=READY pgs=4099 cs=0 l=1 rev1=1 crypto rx=0x7f07380ff440 tx=0x7f072800b410 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:26:12.198 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.198+0000 7f072f7fe640 1 -- 172.21.15.131:0/2931262618 <== mon.2 v2:172.21.15.131:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f073405ad40 con 0x7f0738101470 2024-02-05T08:26:12.373 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:12.373+0000 7f07406ae640 1 -- 172.21.15.131:0/2931262618 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0738198270 con 0x7f0738101470 2024-02-05T08:26:13.049 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.049+0000 7f072f7fe640 1 -- 172.21.15.131:0/2931262618 <== mon.2 v2:172.21.15.131:3301/0 7 ==== osd_map(792..792 src has 254..792) v4 ==== 296+0+0 (secure 0 0 0) 0x7f0734003780 con 0x7f0738101470 2024-02-05T08:26:13.049 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.049+0000 7f072f7fe640 1 -- 172.21.15.131:0/2931262618 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_subscribe({osdmap=793}) v3 -- 0x7f072407a030 con 0x7f0738101470 2024-02-05T08:26:13.056 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.056+0000 7f072f7fe640 1 -- 172.21.15.131:0/2931262618 <== mon.2 v2:172.21.15.131:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]=0 pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' removed v792) v1 ==== 248+0+0 (secure 0 0 0) 0x7f0738198270 con 0x7f0738101470 2024-02-05T08:26:13.165 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.165+0000 7f07406ae640 1 -- 172.21.15.131:0/2931262618 --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0738110a30 con 0x7f0738101470 2024-02-05T08:26:13.165 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.166+0000 7f072f7fe640 1 -- 172.21.15.131:0/2931262618 <== mon.2 v2:172.21.15.131:3301/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]=0 pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' does not exist v792) v1 ==== 255+0+0 (secure 0 0 0) 0x7f073405a8d0 con 0x7f0738101470 2024-02-05T08:26:13.165 INFO:tasks.workunit.client.0.smithi131.stderr:pool 'b9122de8-c2e1-4670-8261-fe325aa3ccfb' does not exist 2024-02-05T08:26:13.168 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.167+0000 7f07406ae640 1 -- 172.21.15.131:0/2931262618 >> v2:172.21.15.131:6800/2614238408 conn(0x7f072406d9f0 msgr2=0x7f072406feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:26:13.168 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.167+0000 7f07406ae640 1 --2- 172.21.15.131:0/2931262618 >> v2:172.21.15.131:6800/2614238408 conn(0x7f072406d9f0 0x7f072406feb0 secure :-1 s=READY pgs=4099 cs=0 l=1 rev1=1 crypto rx=0x7f07380ff440 tx=0x7f072800b410 comp rx=0 tx=0).stop 2024-02-05T08:26:13.168 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.167+0000 7f07406ae640 1 -- 172.21.15.131:0/2931262618 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f0738101470 msgr2=0x7f0738196f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:26:13.168 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.167+0000 7f07406ae640 1 --2- 172.21.15.131:0/2931262618 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f0738101470 0x7f0738196f20 secure :-1 s=READY pgs=3116 cs=0 l=1 rev1=1 crypto rx=0x7f073400ea60 tx=0x7f073400ef20 comp rx=0 tx=0).stop 2024-02-05T08:26:13.168 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.167+0000 7f07406ae640 1 -- 172.21.15.131:0/2931262618 shutdown_connections 2024-02-05T08:26:13.168 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.167+0000 7f07406ae640 1 --2- 172.21.15.131:0/2931262618 >> v2:172.21.15.131:6800/2614238408 conn(0x7f072406d9f0 0x7f072406feb0 unknown :-1 s=CLOSED pgs=4099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:13.168 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.167+0000 7f07406ae640 1 --2- 172.21.15.131:0/2931262618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f073819b7e0 0x7f073819bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:13.168 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.167+0000 7f07406ae640 1 --2- 172.21.15.131:0/2931262618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0738106540 0x7f0738197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:13.168 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.167+0000 7f07406ae640 1 --2- 172.21.15.131:0/2931262618 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7f0738101470 0x7f0738196f20 unknown :-1 s=CLOSED pgs=3116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:13.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.168+0000 7f07406ae640 1 -- 172.21.15.131:0/2931262618 >> 172.21.15.131:0/2931262618 conn(0x7f07380fcd30 msgr2=0x7f07380fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:26:13.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.168+0000 7f07406ae640 1 -- 172.21.15.131:0/2931262618 shutdown_connections 2024-02-05T08:26:13.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.168+0000 7f07406ae640 1 -- 172.21.15.131:0/2931262618 wait complete. 2024-02-05T08:26:13.177 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool delete 264c4d54-5197-4f45-8a91-42b2f8b06ac6 264c4d54-5197-4f45-8a91-42b2f8b06ac6 --yes-i-really-really-mean-it 2024-02-05T08:26:13.261 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.259+0000 7ff564ea8640 1 -- 172.21.15.131:0/1438360918 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff560100d10 msgr2=0x7ff560104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:26:13.261 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.259+0000 7ff564ea8640 1 --2- 172.21.15.131:0/1438360918 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff560100d10 0x7ff560104d50 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 crypto rx=0x7ff554009900 tx=0x7ff55402bc30 comp rx=0 tx=0).stop 2024-02-05T08:26:13.261 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.260+0000 7ff564ea8640 1 -- 172.21.15.131:0/1438360918 shutdown_connections 2024-02-05T08:26:13.261 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.260+0000 7ff564ea8640 1 --2- 172.21.15.131:0/1438360918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff560105430 0x7ff56010d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:13.261 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.260+0000 7ff564ea8640 1 --2- 172.21.15.131:0/1438360918 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff560100d10 0x7ff560104d50 unknown :-1 s=CLOSED pgs=3117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:13.261 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.260+0000 7ff564ea8640 1 --2- 172.21.15.131:0/1438360918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff560100360 0x7ff560100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:13.261 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.260+0000 7ff564ea8640 1 -- 172.21.15.131:0/1438360918 >> 172.21.15.131:0/1438360918 conn(0x7ff560077250 msgr2=0x7ff560077660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:26:13.261 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.260+0000 7ff564ea8640 1 -- 172.21.15.131:0/1438360918 shutdown_connections 2024-02-05T08:26:13.261 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.260+0000 7ff564ea8640 1 -- 172.21.15.131:0/1438360918 wait complete. 2024-02-05T08:26:13.264 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.260+0000 7ff564ea8640 1 Processor -- start 2024-02-05T08:26:13.264 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.260+0000 7ff564ea8640 1 -- start start 2024-02-05T08:26:13.264 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff564ea8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff560100360 0x7ff560196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:26:13.264 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff564ea8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff560105430 0x7ff5601974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:26:13.264 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff564ea8640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff56019b7e0 0x7ff56019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:26:13.265 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff564ea8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56019c1d0 con 0x7ff560100360 2024-02-05T08:26:13.265 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff564ea8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56019c340 con 0x7ff560105430 2024-02-05T08:26:13.265 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff564ea8640 1 -- --> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56019c4b0 con 0x7ff56019b7e0 2024-02-05T08:26:13.265 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff55e59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff560100360 0x7ff560196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:26:13.265 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff55ed9d640 1 --2- >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff56019b7e0 0x7ff56019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:26:13.265 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff55e59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff560100360 0x7ff560196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52754/0 (socket says 172.21.15.131:52754) 2024-02-05T08:26:13.265 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff55e59c640 1 -- 172.21.15.131:0/2266163438 learned_addr learned my addr 172.21.15.131:0/2266163438 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-02-05T08:26:13.265 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff55dd9b640 1 --2- 172.21.15.131:0/2266163438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff560105430 0x7ff5601974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:26:13.265 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff55e59c640 1 -- 172.21.15.131:0/2266163438 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff56019b7e0 msgr2=0x7ff56019bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:26:13.265 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff55e59c640 1 --2- 172.21.15.131:0/2266163438 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff56019b7e0 0x7ff56019bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:13.265 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff55e59c640 1 -- 172.21.15.131:0/2266163438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff560105430 msgr2=0x7ff5601974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:26:13.265 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff55e59c640 1 --2- 172.21.15.131:0/2266163438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff560105430 0x7ff5601974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:13.265 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff55e59c640 1 -- 172.21.15.131:0/2266163438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff558009490 con 0x7ff560100360 2024-02-05T08:26:13.265 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff55ed9d640 1 --2- 172.21.15.131:0/2266163438 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff56019b7e0 0x7ff56019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:26:13.265 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff55dd9b640 1 --2- 172.21.15.131:0/2266163438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff560105430 0x7ff5601974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T08:26:13.266 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff55e59c640 1 --2- 172.21.15.131:0/2266163438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff560100360 0x7ff560196f70 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 crypto rx=0x7ff55000cce0 tx=0x7ff55000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:26:13.266 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff54f7fe640 1 -- 172.21.15.131:0/2266163438 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55000f450 con 0x7ff560100360 2024-02-05T08:26:13.266 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff564ea8640 1 -- 172.21.15.131:0/2266163438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff554009560 con 0x7ff560100360 2024-02-05T08:26:13.266 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff54f7fe640 1 -- 172.21.15.131:0/2266163438 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7ff55000fa60 con 0x7ff560100360 2024-02-05T08:26:13.266 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff54f7fe640 1 -- 172.21.15.131:0/2266163438 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff550018a30 con 0x7ff560100360 2024-02-05T08:26:13.266 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.261+0000 7ff564ea8640 1 -- 172.21.15.131:0/2266163438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5601a3dd0 con 0x7ff560100360 2024-02-05T08:26:13.266 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.262+0000 7ff564ea8640 1 -- 172.21.15.131:0/2266163438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff560068110 con 0x7ff560100360 2024-02-05T08:26:13.266 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.263+0000 7ff54f7fe640 1 -- 172.21.15.131:0/2266163438 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7ff550018b90 con 0x7ff560100360 2024-02-05T08:26:13.266 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.263+0000 7ff54f7fe640 1 --2- 172.21.15.131:0/2266163438 >> v2:172.21.15.131:6800/2614238408 conn(0x7ff54806d9f0 0x7ff54806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:26:13.266 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.263+0000 7ff55dd9b640 1 --2- 172.21.15.131:0/2266163438 >> v2:172.21.15.131:6800/2614238408 conn(0x7ff54806d9f0 0x7ff54806feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:26:13.266 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.263+0000 7ff54f7fe640 1 -- 172.21.15.131:0/2266163438 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(792..792 src has 254..792) v4 ==== 7814+0+0 (secure 0 0 0) 0x7ff55008ded0 con 0x7ff560100360 2024-02-05T08:26:13.266 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.263+0000 7ff54f7fe640 1 -- 172.21.15.131:0/2266163438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=793}) v3 -- 0x7ff548078f40 con 0x7ff560100360 2024-02-05T08:26:13.266 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.264+0000 7ff55dd9b640 1 --2- 172.21.15.131:0/2266163438 >> v2:172.21.15.131:6800/2614238408 conn(0x7ff54806d9f0 0x7ff54806feb0 secure :-1 s=READY pgs=4100 cs=0 l=1 rev1=1 crypto rx=0x7ff554006010 tx=0x7ff55402c9d0 comp rx=0 tx=0).ready entity=mgr.14571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:26:13.267 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:26:12 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: ::ffff:172.21.15.157 - - [05/Feb/2024:08:26:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:26:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:13 smithi131 ceph-mon[41654]: pgmap v1793: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:26:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:13 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2931262618' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:13.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:13 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:13.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:13 smithi131 ceph-mon[29507]: pgmap v1793: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:26:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:13 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2931262618' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:13.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:13 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:13.268 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.268+0000 7ff54f7fe640 1 -- 172.21.15.131:0/2266163438 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff55005ace0 con 0x7ff560100360 2024-02-05T08:26:13.442 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:13.442+0000 7ff564ea8640 1 -- 172.21.15.131:0/2266163438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pool2": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff560100d10 con 0x7ff560100360 2024-02-05T08:26:13.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:13 smithi157 ceph-mon[33345]: pgmap v1793: 188 pgs: 188 active+clean; 863 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:26:13.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:13 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2931262618' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:13.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:13 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:14.054 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.054+0000 7ff54f7fe640 1 -- 172.21.15.131:0/2266163438 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pool2": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "yes_i_really_really_mean_it": true}]=0 pool '264c4d54-5197-4f45-8a91-42b2f8b06ac6' removed v793) v1 ==== 248+0+0 (secure 0 0 0) 0x7ff55005a870 con 0x7ff560100360 2024-02-05T08:26:14.055 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.055+0000 7ff54f7fe640 1 -- 172.21.15.131:0/2266163438 <== mon.0 v2:172.21.15.131:3300/0 8 ==== osd_map(793..793 src has 254..793) v4 ==== 296+0+0 (secure 0 0 0) 0x7ff550018e40 con 0x7ff560100360 2024-02-05T08:26:14.055 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.055+0000 7ff54f7fe640 1 -- 172.21.15.131:0/2266163438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=794}) v3 -- 0x7ff548079be0 con 0x7ff560100360 2024-02-05T08:26:14.167 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.167+0000 7ff564ea8640 1 -- 172.21.15.131:0/2266163438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pool2": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff56010f8e0 con 0x7ff560100360 2024-02-05T08:26:14.167 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.167+0000 7ff54f7fe640 1 -- 172.21.15.131:0/2266163438 <== mon.0 v2:172.21.15.131:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pool2": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "yes_i_really_really_mean_it": true}]=0 pool '264c4d54-5197-4f45-8a91-42b2f8b06ac6' does not exist v793) v1 ==== 255+0+0 (secure 0 0 0) 0x7ff55005aa50 con 0x7ff560100360 2024-02-05T08:26:14.167 INFO:tasks.workunit.client.0.smithi131.stderr:pool '264c4d54-5197-4f45-8a91-42b2f8b06ac6' does not exist 2024-02-05T08:26:14.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.169+0000 7ff564ea8640 1 -- 172.21.15.131:0/2266163438 >> v2:172.21.15.131:6800/2614238408 conn(0x7ff54806d9f0 msgr2=0x7ff54806feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:26:14.169 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.169+0000 7ff564ea8640 1 --2- 172.21.15.131:0/2266163438 >> v2:172.21.15.131:6800/2614238408 conn(0x7ff54806d9f0 0x7ff54806feb0 secure :-1 s=READY pgs=4100 cs=0 l=1 rev1=1 crypto rx=0x7ff554006010 tx=0x7ff55402c9d0 comp rx=0 tx=0).stop 2024-02-05T08:26:14.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.169+0000 7ff564ea8640 1 -- 172.21.15.131:0/2266163438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff560100360 msgr2=0x7ff560196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:26:14.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.169+0000 7ff564ea8640 1 --2- 172.21.15.131:0/2266163438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff560100360 0x7ff560196f70 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 crypto rx=0x7ff55000cce0 tx=0x7ff55000a620 comp rx=0 tx=0).stop 2024-02-05T08:26:14.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.169+0000 7ff564ea8640 1 -- 172.21.15.131:0/2266163438 shutdown_connections 2024-02-05T08:26:14.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.169+0000 7ff564ea8640 1 --2- 172.21.15.131:0/2266163438 >> v2:172.21.15.131:6800/2614238408 conn(0x7ff54806d9f0 0x7ff54806feb0 unknown :-1 s=CLOSED pgs=4100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:14.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.169+0000 7ff564ea8640 1 --2- 172.21.15.131:0/2266163438 >> [v2:172.21.15.131:3301/0,v1:172.21.15.131:6790/0] conn(0x7ff56019b7e0 0x7ff56019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:14.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.169+0000 7ff564ea8640 1 --2- 172.21.15.131:0/2266163438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff560105430 0x7ff5601974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:14.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.169+0000 7ff564ea8640 1 --2- 172.21.15.131:0/2266163438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff560100360 0x7ff560196f70 unknown :-1 s=CLOSED pgs=3113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:26:14.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.169+0000 7ff564ea8640 1 -- 172.21.15.131:0/2266163438 >> 172.21.15.131:0/2266163438 conn(0x7ff560077250 msgr2=0x7ff560103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:26:14.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.169+0000 7ff564ea8640 1 -- 172.21.15.131:0/2266163438 shutdown_connections 2024-02-05T08:26:14.170 INFO:tasks.workunit.client.0.smithi131.stderr:2024-02-05T08:26:14.169+0000 7ff564ea8640 1 -- 172.21.15.131:0/2266163438 wait complete. 2024-02-05T08:26:14.179 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo OK 2024-02-05T08:26:14.180 INFO:tasks.workunit.client.0.smithi131.stdout:OK 2024-02-05T08:26:14.180 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-05T08:26:14.180 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-05T08:26:14.249 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-02-05T08:26:14.249 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-05T08:26:14.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T08:26:14.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:14 smithi131 ceph-mon[41654]: osdmap e792: 8 total, 8 up, 8 in 2024-02-05T08:26:14.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2931262618' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:14.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:14 smithi131 ceph-mon[41654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:14.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:14 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2266163438' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pool2": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:14.313 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T08:26:14.313 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:14 smithi131 ceph-mon[29507]: osdmap e792: 8 total, 8 up, 8 in 2024-02-05T08:26:14.313 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2931262618' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:14.314 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:14 smithi131 ceph-mon[29507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:14.314 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:14 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2266163438' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pool2": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:14.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T08:26:14.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:14 smithi157 ceph-mon[33345]: osdmap e792: 8 total, 8 up, 8 in 2024-02-05T08:26:14.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2931262618' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:14.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:14 smithi157 ceph-mon[33345]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "pool2": "b9122de8-c2e1-4670-8261-fe325aa3ccfb", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:14.471 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:14 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2266163438' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pool2": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:14.554 DEBUG:teuthology.parallel:result is None 2024-02-05T08:26:14.555 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-05T08:26:14.621 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-05T08:26:14.621 DEBUG:teuthology.orchestra.run.smithi131:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-02-05T08:26:14.686 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-02-05T08:26:14.687 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-05T08:26:14.701 INFO:tasks.cephadm:Teardown begin 2024-02-05T08:26:14.701 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-05T08:26:14.753 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-05T08:26:14.789 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-05T08:26:14.789 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-05T08:26:14.809 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-05T08:26:14.846 INFO:tasks.cephadm:Stopping all daemons... 2024-02-05T08:26:14.846 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-05T08:26:14.847 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mon.a 2024-02-05T08:26:15.178 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:15 smithi131 ceph-mon[41654]: pgmap v1795: 176 pgs: 176 active+clean; 695 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:26:15.178 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:15 smithi131 ceph-mon[41654]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:26:15.178 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2266163438' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pool2": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T08:26:15.178 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:15 smithi131 ceph-mon[41654]: osdmap e793: 8 total, 8 up, 8 in 2024-02-05T08:26:15.178 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:15 smithi131 ceph-mon[41654]: from='client.? 172.21.15.131:0/2266163438' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pool2": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:15.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:14 smithi131 systemd[1]: Stopping Ceph mon.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:26:15.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:15 smithi131 ceph-mon[29507]: pgmap v1795: 176 pgs: 176 active+clean; 695 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:26:15.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:15 smithi131 ceph-mon[29507]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:26:15.179 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2266163438' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pool2": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T08:26:15.179 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:15 smithi131 ceph-mon[29507]: osdmap e793: 8 total, 8 up, 8 in 2024-02-05T08:26:15.179 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:15 smithi131 ceph-mon[29507]: from='client.? 172.21.15.131:0/2266163438' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pool2": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:15 smithi157 ceph-mon[33345]: pgmap v1795: 176 pgs: 176 active+clean; 695 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:26:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:15 smithi157 ceph-mon[33345]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:26:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2266163438' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pool2": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T08:26:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:15 smithi157 ceph-mon[33345]: osdmap e793: 8 total, 8 up, 8 in 2024-02-05T08:26:15.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:15 smithi157 ceph-mon[33345]: from='client.? 172.21.15.131:0/2266163438' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "pool2": "264c4d54-5197-4f45-8a91-42b2f8b06ac6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:26:15.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:15 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a[29484]: 2024-02-05T08:26:15.177+0000 7fcbe280f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:26:15.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:15 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a[29484]: 2024-02-05T08:26:15.177+0000 7fcbe280f700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-05T08:26:15.517 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:15 smithi131 podman[160099]: 2024-02-05 08:26:15.229125787 +0000 UTC m=+0.194553185 container died 4dc9ef72f635b54c7a848860dca95cb156f97b215dc909a1d35e429e6a9afbcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-05T08:26:15.976 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:15 smithi131 podman[160099]: 2024-02-05 08:26:15.700215635 +0000 UTC m=+0.665643030 container cleanup 4dc9ef72f635b54c7a848860dca95cb156f97b215dc909a1d35e429e6a9afbcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T08:26:15.976 INFO:journalctl@ceph.mon.a.smithi131.stdout:Feb 05 08:26:15 smithi131 bash[160099]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-a 2024-02-05T08:26:16.111 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mon.a.service' 2024-02-05T08:26:16.150 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:26:16.150 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-05T08:26:16.150 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-02-05T08:26:16.150 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mon.c 2024-02-05T08:26:16.482 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:16 smithi131 systemd[1]: Stopping Ceph mon.c for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:26:16.482 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:16 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-c[41631]: 2024-02-05T08:26:16.481+0000 7f06b7e22700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:26:16.482 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:16 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-c[41631]: 2024-02-05T08:26:16.481+0000 7f06b7e22700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-02-05T08:26:16.767 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:16 smithi131 podman[160233]: 2024-02-05 08:26:16.523584059 +0000 UTC m=+0.151960539 container died 72bec76edf25c9a73252ee7790d77c8c2e8eafe79d34854d2b931bf13ec48a24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-c, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T08:26:16.767 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:26:16 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:08:26:16] ENGINE Bus STOPPING 2024-02-05T08:26:16.768 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:26:16 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:08:26:16] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-05T08:26:16.768 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:26:16 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:08:26:16] ENGINE Bus STOPPED 2024-02-05T08:26:16.768 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:26:16 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:08:26:16] ENGINE Bus STARTING 2024-02-05T08:26:16.768 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:26:16 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:08:26:16] ENGINE Serving on http://:::9283 2024-02-05T08:26:16.768 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:26:16 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: [05/Feb/2024:08:26:16] ENGINE Bus STARTED 2024-02-05T08:26:17.147 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:16 smithi131 podman[160233]: 2024-02-05 08:26:16.897167164 +0000 UTC m=+0.525543640 container cleanup 72bec76edf25c9a73252ee7790d77c8c2e8eafe79d34854d2b931bf13ec48a24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-c, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T08:26:17.148 INFO:journalctl@ceph.mon.c.smithi131.stdout:Feb 05 08:26:16 smithi131 bash[160233]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-c 2024-02-05T08:26:17.281 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mon.c.service' 2024-02-05T08:26:17.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:26:17.319 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-02-05T08:26:17.319 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-02-05T08:26:17.319 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mon.b 2024-02-05T08:26:17.624 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:17 smithi157 systemd[1]: Stopping Ceph mon.b for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:26:17.900 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:17 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-b[33321]: 2024-02-05T08:26:17.623+0000 7fafdf14f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:26:17.900 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:17 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-b[33321]: 2024-02-05T08:26:17.623+0000 7fafdf14f700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-02-05T08:26:17.900 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:17 smithi157 podman[79552]: 2024-02-05 08:26:17.900129127 +0000 UTC m=+0.376372984 container died 7f925c7ccbe6c874617c07cf7084e941131defcee925356a1c7c63db80854266 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-b, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240131) 2024-02-05T08:26:18.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:18 smithi157 podman[79552]: 2024-02-05 08:26:18.380499349 +0000 UTC m=+0.856743207 container cleanup 7f925c7ccbe6c874617c07cf7084e941131defcee925356a1c7c63db80854266 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-b, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-05T08:26:18.470 INFO:journalctl@ceph.mon.b.smithi157.stdout:Feb 05 08:26:18 smithi157 bash[79552]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mon-b 2024-02-05T08:26:18.798 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mon.b.service' 2024-02-05T08:26:18.844 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:26:18.844 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-02-05T08:26:18.844 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-05T08:26:18.845 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mgr.y 2024-02-05T08:26:19.162 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:26:18 smithi131 systemd[1]: Stopping Ceph mgr.y for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:26:19.469 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:26:19 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T08:26:19.161+0000 7f36e2c2e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:26:19.469 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:26:19 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y[29767]: 2024-02-05T08:26:19.161+0000 7f36e2c2e700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-05T08:26:19.469 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:26:19 smithi131 podman[160375]: 2024-02-05 08:26:19.193702701 +0000 UTC m=+0.160720676 container died afc5c66e51ee13b90ceb6967acb93cbd2902010f2fb0fa9d0e5b4c8ef7916506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, org.label-schema.schema-version=1.0) 2024-02-05T08:26:19.767 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:26:19 smithi131 podman[160375]: 2024-02-05 08:26:19.568763848 +0000 UTC m=+0.535781821 container cleanup afc5c66e51ee13b90ceb6967acb93cbd2902010f2fb0fa9d0e5b4c8ef7916506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-05T08:26:19.767 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Feb 05 08:26:19 smithi131 bash[160375]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-y 2024-02-05T08:26:19.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mgr.y.service' 2024-02-05T08:26:19.994 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:26:19.994 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-05T08:26:19.994 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-05T08:26:19.994 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mgr.x 2024-02-05T08:26:20.325 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 08:26:20 smithi157 systemd[1]: Stopping Ceph mgr.x for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:26:20.618 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 08:26:20 smithi157 podman[79685]: 2024-02-05 08:26:20.325487951 +0000 UTC m=+0.132690484 container stop 36d1da2f2cb7f7de54a1da26484926e03002d1008ed0e1b5e056bd14bcbd9680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T08:26:20.619 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 08:26:20 smithi157 podman[79685]: 2024-02-05 08:26:20.351499223 +0000 UTC m=+0.158701755 container died 36d1da2f2cb7f7de54a1da26484926e03002d1008ed0e1b5e056bd14bcbd9680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-05T08:26:20.877 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 08:26:20 smithi157 podman[79685]: 2024-02-05 08:26:20.701455063 +0000 UTC m=+0.508657594 container cleanup 36d1da2f2cb7f7de54a1da26484926e03002d1008ed0e1b5e056bd14bcbd9680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-05T08:26:20.877 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 08:26:20 smithi157 bash[79685]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x 2024-02-05T08:26:21.110 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mgr.x.service' 2024-02-05T08:26:21.134 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 08:26:20 smithi157 podman[79704]: 2024-02-05 08:26:20.877840496 +0000 UTC m=+0.547162007 container remove 36d1da2f2cb7f7de54a1da26484926e03002d1008ed0e1b5e056bd14bcbd9680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2) 2024-02-05T08:26:21.134 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 08:26:20 smithi157 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-02-05T08:26:21.134 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 08:26:21 smithi157 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mgr.x.service: Failed with result 'exit-code'. 2024-02-05T08:26:21.134 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 08:26:21 smithi157 systemd[1]: Stopped Ceph mgr.x for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T08:26:21.134 INFO:journalctl@ceph.mgr.x.smithi157.stdout:Feb 05 08:26:21 smithi157 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@mgr.x.service: Consumed 34.745s CPU time. 2024-02-05T08:26:21.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:26:21.147 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-05T08:26:21.147 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-05T08:26:21.147 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.0 2024-02-05T08:26:21.467 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 08:26:21 smithi131 systemd[1]: Stopping Ceph osd.0 for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:26:21.767 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 08:26:21 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-0[46056]: 2024-02-05T08:26:21.467+0000 7fd25f863700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:26:21.767 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 08:26:21 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-0[46056]: 2024-02-05T08:26:21.467+0000 7fd25f863700 -1 osd.0 793 *** Got signal Terminated *** 2024-02-05T08:26:21.767 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 08:26:21 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-0[46056]: 2024-02-05T08:26:21.467+0000 7fd25f863700 -1 osd.0 793 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T08:26:27.204 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 08:26:26 smithi131 podman[160509]: 2024-02-05 08:26:26.941442211 +0000 UTC m=+5.602259371 container died 0e7fd2c67c922cae61817888018bc9ea6e142b6f369af89b61dedb5b080a9cce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-05T08:26:27.517 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 08:26:27 smithi131 podman[160509]: 2024-02-05 08:26:27.314789674 +0000 UTC m=+5.975606838 container cleanup 0e7fd2c67c922cae61817888018bc9ea6e142b6f369af89b61dedb5b080a9cce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-0, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-05T08:26:27.517 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 08:26:27 smithi131 bash[160509]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-0 2024-02-05T08:26:27.867 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 08:26:27 smithi131 podman[160530]: 2024-02-05 08:26:27.574243699 +0000 UTC m=+0.633196710 container remove 0e7fd2c67c922cae61817888018bc9ea6e142b6f369af89b61dedb5b080a9cce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_BRANCH=HEAD) 2024-02-05T08:26:28.161 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 08:26:27 smithi131 podman[160627]: 2024-02-05 08:26:27.770066827 +0000 UTC m=+0.017228569 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T08:26:28.161 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 08:26:27 smithi131 podman[160627]: 2024-02-05 08:26:27.90044428 +0000 UTC m=+0.147606062 container create 204977df7a5482034eaf5b566f024449a422caec374cd53536be1706dbca0250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD) 2024-02-05T08:26:28.455 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 08:26:28 smithi131 podman[160627]: 2024-02-05 08:26:28.161573072 +0000 UTC m=+0.408734809 container init 204977df7a5482034eaf5b566f024449a422caec374cd53536be1706dbca0250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:26:28.456 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 08:26:28 smithi131 podman[160627]: 2024-02-05 08:26:28.167099697 +0000 UTC m=+0.414261437 container start 204977df7a5482034eaf5b566f024449a422caec374cd53536be1706dbca0250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T08:26:28.456 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 08:26:28 smithi131 podman[160627]: 2024-02-05 08:26:28.253101922 +0000 UTC m=+0.500263669 container attach 204977df7a5482034eaf5b566f024449a422caec374cd53536be1706dbca0250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-0-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-05T08:26:28.766 INFO:journalctl@ceph.osd.0.smithi131.stdout:Feb 05 08:26:28 smithi131 podman[160627]: 2024-02-05 08:26:28.456406809 +0000 UTC m=+0.703568555 container died 204977df7a5482034eaf5b566f024449a422caec374cd53536be1706dbca0250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:26:28.895 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.0.service' 2024-02-05T08:26:28.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:26:28.935 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-05T08:26:28.935 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-05T08:26:28.935 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.1 2024-02-05T08:26:29.267 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 08:26:29 smithi131 systemd[1]: Stopping Ceph osd.1 for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:26:29.767 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 08:26:29 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-1[52280]: 2024-02-05T08:26:29.305+0000 7fe27bea9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:26:29.767 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 08:26:29 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-1[52280]: 2024-02-05T08:26:29.305+0000 7fe27bea9700 -1 osd.1 793 *** Got signal Terminated *** 2024-02-05T08:26:29.767 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 08:26:29 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-1[52280]: 2024-02-05T08:26:29.305+0000 7fe27bea9700 -1 osd.1 793 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T08:26:34.968 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 08:26:34 smithi131 podman[160759]: 2024-02-05 08:26:34.681419908 +0000 UTC m=+5.501827026 container died 2b17d6313ba24c9df22c8582aada699764fb2b5b56aebd63d03823946aa2b614 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:26:35.267 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 08:26:35 smithi131 podman[160759]: 2024-02-05 08:26:35.067528368 +0000 UTC m=+5.887935488 container cleanup 2b17d6313ba24c9df22c8582aada699764fb2b5b56aebd63d03823946aa2b614 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:26:35.267 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 08:26:35 smithi131 bash[160759]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-1 2024-02-05T08:26:35.682 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 08:26:35 smithi131 podman[160793]: 2024-02-05 08:26:35.344033752 +0000 UTC m=+0.660330399 container remove 2b17d6313ba24c9df22c8582aada699764fb2b5b56aebd63d03823946aa2b614 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-1, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-05T08:26:36.014 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 08:26:35 smithi131 podman[160887]: 2024-02-05 08:26:35.58716069 +0000 UTC m=+0.019617115 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T08:26:36.014 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 08:26:35 smithi131 podman[160887]: 2024-02-05 08:26:35.703414627 +0000 UTC m=+0.135871049 container create 017f575f14965ca28b7f15875723c26dec9432f6007453bd39ba691cc0d57b0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0) 2024-02-05T08:26:36.014 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 08:26:35 smithi131 podman[160887]: 2024-02-05 08:26:35.897663774 +0000 UTC m=+0.330120191 container init 017f575f14965ca28b7f15875723c26dec9432f6007453bd39ba691cc0d57b0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0) 2024-02-05T08:26:36.014 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 08:26:35 smithi131 podman[160887]: 2024-02-05 08:26:35.902747548 +0000 UTC m=+0.335203965 container start 017f575f14965ca28b7f15875723c26dec9432f6007453bd39ba691cc0d57b0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-1-deactivate, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:26:36.014 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 08:26:36 smithi131 podman[160887]: 2024-02-05 08:26:36.01425323 +0000 UTC m=+0.446709658 container attach 017f575f14965ca28b7f15875723c26dec9432f6007453bd39ba691cc0d57b0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-1-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-05T08:26:36.267 INFO:journalctl@ceph.osd.1.smithi131.stdout:Feb 05 08:26:36 smithi131 podman[160887]: 2024-02-05 08:26:36.196139274 +0000 UTC m=+0.628595692 container died 017f575f14965ca28b7f15875723c26dec9432f6007453bd39ba691cc0d57b0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_BRANCH=HEAD, io.buildah.version=1.33.2) 2024-02-05T08:26:36.836 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.1.service' 2024-02-05T08:26:36.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:26:36.878 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-05T08:26:36.878 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-05T08:26:36.879 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.2 2024-02-05T08:26:37.215 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:36 smithi131 systemd[1]: Stopping Ceph osd.2 for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:26:37.517 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:37 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2[58548]: 2024-02-05T08:26:37.215+0000 7f9767d0d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:26:37.517 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:37 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2[58548]: 2024-02-05T08:26:37.215+0000 7f9767d0d700 -1 osd.2 793 *** Got signal Terminated *** 2024-02-05T08:26:37.517 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:37 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2[58548]: 2024-02-05T08:26:37.215+0000 7f9767d0d700 -1 osd.2 793 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T08:26:42.796 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:42 smithi131 podman[161015]: 2024-02-05 08:26:42.543337911 +0000 UTC m=+5.438806574 container died 876604ec5a4ae156ef5acf657380b2962a872c64d57e6ac29d151b5e574a10ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-02-05T08:26:43.096 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:42 smithi131 podman[161015]: 2024-02-05 08:26:42.862088807 +0000 UTC m=+5.757557469 container cleanup 876604ec5a4ae156ef5acf657380b2962a872c64d57e6ac29d151b5e574a10ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131) 2024-02-05T08:26:43.096 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:42 smithi131 bash[161015]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2 2024-02-05T08:26:43.503 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:43 smithi131 podman[161037]: 2024-02-05 08:26:43.096771979 +0000 UTC m=+0.557367564 container remove 876604ec5a4ae156ef5acf657380b2962a872c64d57e6ac29d151b5e574a10ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-05T08:26:43.754 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:43 smithi131 podman[161135]: 2024-02-05 08:26:43.407501297 +0000 UTC m=+0.018077359 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T08:26:43.755 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:43 smithi131 podman[161135]: 2024-02-05 08:26:43.505806248 +0000 UTC m=+0.116382309 container create 7fc95f078bd331b06f71d999c0941a22305375a153cde44a8fd6e949e12ea16e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:26:43.755 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:43 smithi131 podman[161135]: 2024-02-05 08:26:43.750267137 +0000 UTC m=+0.360843198 container init 7fc95f078bd331b06f71d999c0941a22305375a153cde44a8fd6e949e12ea16e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-05T08:26:43.755 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:43 smithi131 podman[161135]: 2024-02-05 08:26:43.755400262 +0000 UTC m=+0.365976324 container start 7fc95f078bd331b06f71d999c0941a22305375a153cde44a8fd6e949e12ea16e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:26:44.017 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:43 smithi131 podman[161135]: 2024-02-05 08:26:43.858669885 +0000 UTC m=+0.469245947 container attach 7fc95f078bd331b06f71d999c0941a22305375a153cde44a8fd6e949e12ea16e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-05T08:26:44.327 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:44 smithi131 podman[161135]: 2024-02-05 08:26:44.040921102 +0000 UTC m=+0.651497164 container died 7fc95f078bd331b06f71d999c0941a22305375a153cde44a8fd6e949e12ea16e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2-deactivate, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-05T08:26:44.578 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:44 smithi131 podman[161181]: 2024-02-05 08:26:44.56192512 +0000 UTC m=+0.511932414 container remove 7fc95f078bd331b06f71d999c0941a22305375a153cde44a8fd6e949e12ea16e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131) 2024-02-05T08:26:44.578 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:44 smithi131 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.2.service: Deactivated successfully. 2024-02-05T08:26:44.578 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:44 smithi131 systemd[1]: Stopped Ceph osd.2 for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T08:26:44.578 INFO:journalctl@ceph.osd.2.smithi131.stdout:Feb 05 08:26:44 smithi131 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.2.service: Consumed 50.286s CPU time. 2024-02-05T08:26:44.579 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.2.service' 2024-02-05T08:26:44.618 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:26:44.619 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-05T08:26:44.619 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-05T08:26:44.619 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.3 2024-02-05T08:26:44.951 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:44 smithi131 systemd[1]: Stopping Ceph osd.3 for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:26:45.267 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:44 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3[65147]: 2024-02-05T08:26:44.949+0000 7f18d55ca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:26:45.267 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:44 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3[65147]: 2024-02-05T08:26:44.949+0000 7f18d55ca700 -1 osd.3 793 *** Got signal Terminated *** 2024-02-05T08:26:45.267 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:44 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3[65147]: 2024-02-05T08:26:44.949+0000 7f18d55ca700 -1 osd.3 793 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T08:26:48.220 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:47 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:47.886+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:26:48.221 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:48 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:48.072+0000 7fbb4e36c700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:24.881373+0000 front 2024-02-05T08:26:24.881530+0000 (oldest deadline 2024-02-05T08:26:47.781419+0000) 2024-02-05T08:26:49.017 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:48 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3[65147]: 2024-02-05T08:26:48.590+0000 7f18cdbbd700 -1 osd.3 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:25.074465+0000 front 2024-02-05T08:26:25.074633+0000 (oldest deadline 2024-02-05T08:26:48.574634+0000) 2024-02-05T08:26:49.220 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:48 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:48.867+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:26:49.220 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:49.093+0000 7fbb4e36c700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:24.881373+0000 front 2024-02-05T08:26:24.881530+0000 (oldest deadline 2024-02-05T08:26:47.781419+0000) 2024-02-05T08:26:50.017 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:49 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3[65147]: 2024-02-05T08:26:49.586+0000 7f18cdbbd700 -1 osd.3 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:25.074465+0000 front 2024-02-05T08:26:25.074633+0000 (oldest deadline 2024-02-05T08:26:48.574634+0000) 2024-02-05T08:26:50.125 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:49 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:49.823+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:26:50.470 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:50 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:50.124+0000 7fbb4e36c700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:24.881373+0000 front 2024-02-05T08:26:24.881530+0000 (oldest deadline 2024-02-05T08:26:47.781419+0000) 2024-02-05T08:26:50.707 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:50 smithi131 podman[161268]: 2024-02-05 08:26:50.437029485 +0000 UTC m=+5.592238479 container died e5a69fb3769ff85378aa52abbad837d7f57c42c271c6d4cc077c8a4fdefcb8fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, ceph=True, GIT_BRANCH=HEAD) 2024-02-05T08:26:51.017 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:50 smithi131 podman[161268]: 2024-02-05 08:26:50.806239028 +0000 UTC m=+5.961448024 container cleanup e5a69fb3769ff85378aa52abbad837d7f57c42c271c6d4cc077c8a4fdefcb8fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_BRANCH=HEAD) 2024-02-05T08:26:51.017 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:50 smithi131 bash[161268]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3 2024-02-05T08:26:51.120 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:50 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:50.863+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:26:51.121 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:26:51 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:26:51.010+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:26:51.433 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:51 smithi131 podman[161290]: 2024-02-05 08:26:51.074499942 +0000 UTC m=+0.632427799 container remove e5a69fb3769ff85378aa52abbad837d7f57c42c271c6d4cc077c8a4fdefcb8fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:26:51.470 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:51 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:51.120+0000 7fbb4e36c700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:24.881373+0000 front 2024-02-05T08:26:24.881530+0000 (oldest deadline 2024-02-05T08:26:47.781419+0000) 2024-02-05T08:26:51.735 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:51 smithi131 podman[161385]: 2024-02-05 08:26:51.336668134 +0000 UTC m=+0.016812566 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T08:26:51.735 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:51 smithi131 podman[161385]: 2024-02-05 08:26:51.441966481 +0000 UTC m=+0.122110955 container create 766a521061a8c955eb41c9b3c3fbd1ae3ca77aee2fe37cc34e5da6ec37a6c5af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:26:51.735 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:51 smithi131 podman[161385]: 2024-02-05 08:26:51.644487947 +0000 UTC m=+0.324632380 container init 766a521061a8c955eb41c9b3c3fbd1ae3ca77aee2fe37cc34e5da6ec37a6c5af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:26:51.735 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:51 smithi131 podman[161385]: 2024-02-05 08:26:51.649893298 +0000 UTC m=+0.330037737 container start 766a521061a8c955eb41c9b3c3fbd1ae3ca77aee2fe37cc34e5da6ec37a6c5af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_CLEAN=True, org.label-schema.build-date=20240131) 2024-02-05T08:26:51.736 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:51 smithi131 podman[161385]: 2024-02-05 08:26:51.735808063 +0000 UTC m=+0.415952512 container attach 766a521061a8c955eb41c9b3c3fbd1ae3ca77aee2fe37cc34e5da6ec37a6c5af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3-deactivate, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:26:52.017 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:51 smithi131 podman[161385]: 2024-02-05 08:26:51.935181138 +0000 UTC m=+0.615325575 container died 766a521061a8c955eb41c9b3c3fbd1ae3ca77aee2fe37cc34e5da6ec37a6c5af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-05T08:26:52.220 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:52 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:52.112+0000 7fbb4e36c700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:24.881373+0000 front 2024-02-05T08:26:24.881530+0000 (oldest deadline 2024-02-05T08:26:47.781419+0000) 2024-02-05T08:26:52.220 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:51 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:51.912+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:26:52.220 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:26:52 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:26:52.033+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:26:52.493 INFO:journalctl@ceph.osd.3.smithi131.stdout:Feb 05 08:26:52 smithi131 podman[161431]: 2024-02-05 08:26:52.472428686 +0000 UTC m=+0.532479993 container remove 766a521061a8c955eb41c9b3c3fbd1ae3ca77aee2fe37cc34e5da6ec37a6c5af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:26:52.499 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.3.service' 2024-02-05T08:26:52.541 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:26:52.541 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-05T08:26:52.541 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-05T08:26:52.541 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.4 2024-02-05T08:26:52.865 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:52 smithi157 systemd[1]: Stopping Ceph osd.4 for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:26:52.865 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:26:52 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:26:52.499+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:26:53.220 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:52 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:52.864+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:26:53.220 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:26:53 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:26:53.079+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:26:53.221 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:53 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:53.085+0000 7fbb4e36c700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:24.881373+0000 front 2024-02-05T08:26:24.881530+0000 (oldest deadline 2024-02-05T08:26:47.781419+0000) 2024-02-05T08:26:53.221 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:53 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:53.089+0000 7fbb51d71700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:26:53.221 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:53 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:53.089+0000 7fbb51d71700 -1 osd.4 793 *** Got signal Terminated *** 2024-02-05T08:26:53.221 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:53 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:53.089+0000 7fbb51d71700 -1 osd.4 793 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T08:26:53.860 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:26:53 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:26:53.482+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:26:54.220 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:54 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:54.070+0000 7fbb4e36c700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:24.881373+0000 front 2024-02-05T08:26:24.881530+0000 (oldest deadline 2024-02-05T08:26:47.781419+0000) 2024-02-05T08:26:54.220 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:53 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:53.860+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:26:54.221 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:26:54 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:26:54.106+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:26:54.841 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:26:54 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:26:54.523+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:26:55.220 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:54 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:54.841+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:26:55.221 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:55.058+0000 7fbb4e36c700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:24.881373+0000 front 2024-02-05T08:26:24.881530+0000 (oldest deadline 2024-02-05T08:26:47.781419+0000) 2024-02-05T08:26:55.221 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:26:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:26:55.077+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:26:55.221 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:26:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:26:55.077+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:26:55.823 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:26:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:26:55.498+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:26:56.097 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:55.823+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:26:56.097 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:55 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:55.823+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:33.454281+0000 front 2024-02-05T08:26:33.454108+0000 (oldest deadline 2024-02-05T08:26:55.654486+0000) 2024-02-05T08:26:56.470 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:56.097+0000 7fbb4e36c700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:24.881373+0000 front 2024-02-05T08:26:24.881530+0000 (oldest deadline 2024-02-05T08:26:47.781419+0000) 2024-02-05T08:26:56.470 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:56.097+0000 7fbb4e36c700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:30.082213+0000 front 2024-02-05T08:26:30.082106+0000 (oldest deadline 2024-02-05T08:26:55.982194+0000) 2024-02-05T08:26:56.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:26:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:26:56.115+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:26:56.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:26:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:26:56.115+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:26:56.856 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:26:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:26:56.500+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:26:57.220 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:57 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:57.065+0000 7fbb4e36c700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:24.881373+0000 front 2024-02-05T08:26:24.881530+0000 (oldest deadline 2024-02-05T08:26:47.781419+0000) 2024-02-05T08:26:57.220 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:57 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:57.065+0000 7fbb4e36c700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:30.082213+0000 front 2024-02-05T08:26:30.082106+0000 (oldest deadline 2024-02-05T08:26:55.982194+0000) 2024-02-05T08:26:57.221 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:56.855+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:26:57.221 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:56.855+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:33.454281+0000 front 2024-02-05T08:26:33.454108+0000 (oldest deadline 2024-02-05T08:26:55.654486+0000) 2024-02-05T08:26:57.221 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:26:57 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:26:57.089+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:26:57.221 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:26:57 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:26:57.089+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:26:57.221 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:26:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T08:26:56.888Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.131:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.131:8765: connect: connection refused" 2024-02-05T08:26:57.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:26:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T08:26:56.888Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.131:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.131:8765: connect: connection refused" 2024-02-05T08:26:57.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:26:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T08:26:56.888Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.131:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.131:8765: connect: connection refused" 2024-02-05T08:26:57.222 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:26:56 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T08:26:56.889Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.131:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.131:8765: connect: connection refused" 2024-02-05T08:26:57.846 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:26:57 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:26:57.503+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:26:57.846 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:26:57 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:26:57.503+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:26:58.220 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:58.086+0000 7fbb4e36c700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:24.881373+0000 front 2024-02-05T08:26:24.881530+0000 (oldest deadline 2024-02-05T08:26:47.781419+0000) 2024-02-05T08:26:58.220 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4[38851]: 2024-02-05T08:26:58.086+0000 7fbb4e36c700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:30.082213+0000 front 2024-02-05T08:26:30.082106+0000 (oldest deadline 2024-02-05T08:26:55.982194+0000) 2024-02-05T08:26:58.221 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:57 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:57.846+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:26:58.221 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:57 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:57.846+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:33.454281+0000 front 2024-02-05T08:26:33.454108+0000 (oldest deadline 2024-02-05T08:26:55.654486+0000) 2024-02-05T08:26:58.221 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:26:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:26:58.077+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:26:58.221 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:26:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:26:58.077+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:26:58.894 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:58 smithi157 podman[79835]: 2024-02-05 08:26:58.637980398 +0000 UTC m=+5.901714719 container died 66f05420f2f15301c91f9957ee45e5caddddfd83f6f956bc2cc737f6fae4aacc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:26:58.894 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:26:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:26:58.505+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:26:58.895 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:26:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:26:58.505+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:26:59.220 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:26:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:26:59.035+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:26:59.220 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:26:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:26:59.035+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:26:59.221 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:59 smithi157 podman[79835]: 2024-02-05 08:26:59.011172059 +0000 UTC m=+6.274906370 container cleanup 66f05420f2f15301c91f9957ee45e5caddddfd83f6f956bc2cc737f6fae4aacc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, org.label-schema.vendor=CentOS) 2024-02-05T08:26:59.221 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:59 smithi157 bash[79835]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4 2024-02-05T08:26:59.221 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:58.894+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:26:59.221 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:58 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:58.894+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:33.454281+0000 front 2024-02-05T08:26:33.454108+0000 (oldest deadline 2024-02-05T08:26:55.654486+0000) 2024-02-05T08:26:59.504 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:59 smithi157 podman[79857]: 2024-02-05 08:26:59.287407413 +0000 UTC m=+0.647493136 container remove 66f05420f2f15301c91f9957ee45e5caddddfd83f6f956bc2cc737f6fae4aacc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-05T08:26:59.840 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:59 smithi157 podman[79953]: 2024-02-05 08:26:59.638482111 +0000 UTC m=+0.114561888 container create 0a72d59715d5d3ab218715b325637f7d73de6a869347a8004515059b493f0523 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-05T08:26:59.840 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:59 smithi157 podman[79953]: 2024-02-05 08:26:59.542299707 +0000 UTC m=+0.018379442 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T08:26:59.841 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:26:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:26:59.504+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:26:59.841 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:26:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:26:59.504+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:00.131 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:00 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:00.078+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:27:00.132 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:00 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:00.078+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:27:00.132 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:59 smithi157 podman[79953]: 2024-02-05 08:26:59.840918938 +0000 UTC m=+0.316998691 container init 0a72d59715d5d3ab218715b325637f7d73de6a869347a8004515059b493f0523 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131) 2024-02-05T08:27:00.132 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:59 smithi157 podman[79953]: 2024-02-05 08:26:59.846597402 +0000 UTC m=+0.322677144 container start 0a72d59715d5d3ab218715b325637f7d73de6a869347a8004515059b493f0523 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:27:00.132 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:26:59 smithi157 podman[79953]: 2024-02-05 08:26:59.95727241 +0000 UTC m=+0.433352157 container attach 0a72d59715d5d3ab218715b325637f7d73de6a869347a8004515059b493f0523 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-02-05T08:27:00.133 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:59.882+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:27:00.133 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:26:59 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:26:59.882+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:33.454281+0000 front 2024-02-05T08:26:33.454108+0000 (oldest deadline 2024-02-05T08:26:55.654486+0000) 2024-02-05T08:27:00.426 INFO:journalctl@ceph.osd.4.smithi157.stdout:Feb 05 08:27:00 smithi157 podman[79953]: 2024-02-05 08:27:00.132389443 +0000 UTC m=+0.608469185 container died 0a72d59715d5d3ab218715b325637f7d73de6a869347a8004515059b493f0523 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-05T08:27:00.720 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:00 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:00.470+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:00.720 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:00 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:00.470+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:00.776 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.4.service' 2024-02-05T08:27:00.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:27:00.820 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-05T08:27:00.820 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-05T08:27:00.820 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.5 2024-02-05T08:27:01.123 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:00 smithi157 systemd[1]: Stopping Ceph osd.5 for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:27:01.123 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:00 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:00.920+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:27:01.124 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:00 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:00.920+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:33.454281+0000 front 2024-02-05T08:26:33.454108+0000 (oldest deadline 2024-02-05T08:26:55.654486+0000) 2024-02-05T08:27:01.470 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:01 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:01.158+0000 7f867cf4b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:27:01.470 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:01 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:01.158+0000 7f867cf4b700 -1 osd.5 793 *** Got signal Terminated *** 2024-02-05T08:27:01.470 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:01 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:01.158+0000 7f867cf4b700 -1 osd.5 793 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T08:27:01.471 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:01 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:01.124+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:27:01.471 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:01 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:01.124+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:27:01.934 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:01 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:01.501+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:01.934 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:01 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:01.501+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:02.220 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:01 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:01.934+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:27:02.220 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:01 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:01.934+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:33.454281+0000 front 2024-02-05T08:26:33.454108+0000 (oldest deadline 2024-02-05T08:26:55.654486+0000) 2024-02-05T08:27:02.220 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:02 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:02.126+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:27:02.220 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:02 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:02.126+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:27:02.922 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:02 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:02.527+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:02.922 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:02 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:02.527+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:03.220 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:02 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:02.922+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:27:03.220 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:02 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:02.922+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:33.454281+0000 front 2024-02-05T08:26:33.454108+0000 (oldest deadline 2024-02-05T08:26:55.654486+0000) 2024-02-05T08:27:03.220 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:03 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:03.152+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:27:03.221 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:03 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:03.152+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:27:03.221 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:03 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:03.152+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:37.184692+0000 front 2024-02-05T08:26:37.184663+0000 (oldest deadline 2024-02-05T08:27:03.084607+0000) 2024-02-05T08:27:03.883 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:03 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:03.541+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:03.884 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:03 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:03.541+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:03.884 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:03 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:03.541+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:04.141 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:03 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:03.883+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:27:04.141 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:03 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:03.883+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:33.454281+0000 front 2024-02-05T08:26:33.454108+0000 (oldest deadline 2024-02-05T08:26:55.654486+0000) 2024-02-05T08:27:04.141 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:03 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:03.883+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:38.455725+0000 front 2024-02-05T08:26:38.455749+0000 (oldest deadline 2024-02-05T08:27:03.155735+0000) 2024-02-05T08:27:04.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:04 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:04.140+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:27:04.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:04 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:04.140+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:27:04.471 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:04 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:04.140+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:37.184692+0000 front 2024-02-05T08:26:37.184663+0000 (oldest deadline 2024-02-05T08:27:03.084607+0000) 2024-02-05T08:27:04.903 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:04 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:04.554+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:04.904 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:04 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:04.554+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:04.904 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:04 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:04.554+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:05.220 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:04 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:04.903+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:27:05.220 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:04 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:04.903+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:33.454281+0000 front 2024-02-05T08:26:33.454108+0000 (oldest deadline 2024-02-05T08:26:55.654486+0000) 2024-02-05T08:27:05.220 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:04 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:04.903+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:38.455725+0000 front 2024-02-05T08:26:38.455749+0000 (oldest deadline 2024-02-05T08:27:03.155735+0000) 2024-02-05T08:27:05.220 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:05 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:05.152+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:27:05.220 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:05 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:05.152+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:27:05.221 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:05 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:05.152+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:37.184692+0000 front 2024-02-05T08:26:37.184663+0000 (oldest deadline 2024-02-05T08:27:03.084607+0000) 2024-02-05T08:27:05.894 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:05 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:05.577+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:05.894 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:05 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:05.577+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:05.894 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:05 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:05.577+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:05.894 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:05 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:05.578+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:06.164 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:05 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:05.893+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:22.952523+0000 front 2024-02-05T08:26:22.952565+0000 (oldest deadline 2024-02-05T08:26:47.652331+0000) 2024-02-05T08:27:06.164 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:05 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:05.893+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:33.454281+0000 front 2024-02-05T08:26:33.454108+0000 (oldest deadline 2024-02-05T08:26:55.654486+0000) 2024-02-05T08:27:06.164 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:05 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5[45210]: 2024-02-05T08:27:05.893+0000 7f867553e700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:38.455725+0000 front 2024-02-05T08:26:38.455749+0000 (oldest deadline 2024-02-05T08:27:03.155735+0000) 2024-02-05T08:27:06.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:06 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:06.164+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:27:06.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:06 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:06.164+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:27:06.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:06 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:06.164+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:37.184692+0000 front 2024-02-05T08:26:37.184663+0000 (oldest deadline 2024-02-05T08:27:03.084607+0000) 2024-02-05T08:27:06.921 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:06 smithi157 podman[80083]: 2024-02-05 08:27:06.640785458 +0000 UTC m=+5.593761054 container died 708697be41686513c5db10bba3caaad11410e8c80e16724a13966b64491a11c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240131) 2024-02-05T08:27:06.921 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:06 smithi157 podman[80083]: 2024-02-05 08:27:06.921481704 +0000 UTC m=+5.874457302 container cleanup 708697be41686513c5db10bba3caaad11410e8c80e16724a13966b64491a11c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-05T08:27:06.921 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:06 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:06.615+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:06.921 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:06 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:06.615+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:06.921 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:06 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:06.615+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:06.921 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:06 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:06.615+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:07.192 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:06 smithi157 bash[80083]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5 2024-02-05T08:27:07.192 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:07 smithi157 podman[80103]: 2024-02-05 08:27:07.164376696 +0000 UTC m=+0.520580396 container remove 708697be41686513c5db10bba3caaad11410e8c80e16724a13966b64491a11c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2) 2024-02-05T08:27:07.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:07 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:07.191+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:27:07.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:07 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:07.191+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:27:07.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:07 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:07.191+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:37.184692+0000 front 2024-02-05T08:26:37.184663+0000 (oldest deadline 2024-02-05T08:27:03.084607+0000) 2024-02-05T08:27:07.734 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:07 smithi157 podman[80204]: 2024-02-05 08:27:07.386630216 +0000 UTC m=+0.018033392 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T08:27:07.734 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:07 smithi157 podman[80204]: 2024-02-05 08:27:07.506823972 +0000 UTC m=+0.138227186 container create 515e0400621c5abe1541b82af8388872319b5d9bf27e8f3278dd05abd5459bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=) 2024-02-05T08:27:07.734 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:07 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:07.576+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:07.734 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:07 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:07.576+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:07.734 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:07 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:07.576+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:07.734 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:07 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:07.576+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:08.034 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:07 smithi157 podman[80204]: 2024-02-05 08:27:07.734210326 +0000 UTC m=+0.365613494 container init 515e0400621c5abe1541b82af8388872319b5d9bf27e8f3278dd05abd5459bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5-deactivate, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131) 2024-02-05T08:27:08.034 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:07 smithi157 podman[80204]: 2024-02-05 08:27:07.739979647 +0000 UTC m=+0.371382815 container start 515e0400621c5abe1541b82af8388872319b5d9bf27e8f3278dd05abd5459bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:27:08.034 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:07 smithi157 podman[80204]: 2024-02-05 08:27:07.834314123 +0000 UTC m=+0.465717293 container attach 515e0400621c5abe1541b82af8388872319b5d9bf27e8f3278dd05abd5459bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-05T08:27:08.034 INFO:journalctl@ceph.osd.5.smithi157.stdout:Feb 05 08:27:08 smithi157 podman[80204]: 2024-02-05 08:27:08.034677544 +0000 UTC m=+0.666080720 container died 515e0400621c5abe1541b82af8388872319b5d9bf27e8f3278dd05abd5459bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-5-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-05T08:27:08.378 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:08.214+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:27:08.378 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:08.214+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:27:08.378 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:08.214+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:37.184692+0000 front 2024-02-05T08:26:37.184663+0000 (oldest deadline 2024-02-05T08:27:03.084607+0000) 2024-02-05T08:27:08.679 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:08.572+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:08.679 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:08.572+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:08.679 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:08.572+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:08.679 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:08 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:08.572+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:08.694 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.5.service' 2024-02-05T08:27:08.736 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:27:08.736 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-05T08:27:08.737 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-05T08:27:08.737 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.6 2024-02-05T08:27:08.970 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:08 smithi157 systemd[1]: Stopping Ceph osd.6 for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:27:09.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:09.051+0000 7f25f1a3d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:27:09.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:09.051+0000 7f25f1a3d700 -1 osd.6 793 *** Got signal Terminated *** 2024-02-05T08:27:09.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:09.051+0000 7f25f1a3d700 -1 osd.6 793 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T08:27:09.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:09.200+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:27:09.471 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:09.200+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:27:09.471 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:09.200+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:37.184692+0000 front 2024-02-05T08:26:37.184663+0000 (oldest deadline 2024-02-05T08:27:03.084607+0000) 2024-02-05T08:27:09.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:09.539+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:09.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:09.539+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:09.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:09.539+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:09.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:09 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:09.540+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:10.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:10 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:10.155+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:27:10.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:10 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:10.155+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:27:10.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:10 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:10.155+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:37.184692+0000 front 2024-02-05T08:26:37.184663+0000 (oldest deadline 2024-02-05T08:27:03.084607+0000) 2024-02-05T08:27:10.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:10 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:10.588+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:10.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:10 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:10.588+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:10.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:10 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:10.588+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:10.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:10 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:10.588+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:11.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:11 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:11.118+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:27:11.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:11 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:11.118+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:27:11.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:11 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:11.118+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:37.184692+0000 front 2024-02-05T08:26:37.184663+0000 (oldest deadline 2024-02-05T08:27:03.084607+0000) 2024-02-05T08:27:11.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:11 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:11.615+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:11.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:11 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:11.615+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:11.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:11 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:11.615+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:11.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:11 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:11.615+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:12.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:12 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:12.150+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:27:12.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:12 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:12.150+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:27:12.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:12 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:12.150+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:37.184692+0000 front 2024-02-05T08:26:37.184663+0000 (oldest deadline 2024-02-05T08:27:03.084607+0000) 2024-02-05T08:27:12.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:12 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:12.150+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-02-05T08:26:46.585632+0000 front 2024-02-05T08:26:46.585642+0000 (oldest deadline 2024-02-05T08:27:11.285588+0000) 2024-02-05T08:27:12.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:12 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:12.627+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:12.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:12 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:12.627+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:12.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:12 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:12.627+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:12.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:12 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:12.627+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-02-05T08:26:49.413580+0000 front 2024-02-05T08:26:49.413531+0000 (oldest deadline 2024-02-05T08:27:12.313452+0000) 2024-02-05T08:27:12.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:12 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:12.628+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:13.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:13 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:13.156+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.182830+0000 front 2024-02-05T08:26:26.182947+0000 (oldest deadline 2024-02-05T08:26:50.282973+0000) 2024-02-05T08:27:13.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:13 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:13.156+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:31.983896+0000 front 2024-02-05T08:26:31.983914+0000 (oldest deadline 2024-02-05T08:26:54.883786+0000) 2024-02-05T08:27:13.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:13 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:13.156+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:37.184692+0000 front 2024-02-05T08:26:37.184663+0000 (oldest deadline 2024-02-05T08:27:03.084607+0000) 2024-02-05T08:27:13.470 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:13 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6[51290]: 2024-02-05T08:27:13.156+0000 7f25ea030700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-02-05T08:26:46.585632+0000 front 2024-02-05T08:26:46.585642+0000 (oldest deadline 2024-02-05T08:27:11.285588+0000) 2024-02-05T08:27:13.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:13 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:13.625+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:13.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:13 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:13.625+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:13.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:13 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:13.625+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:13.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:13 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:13.625+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-02-05T08:26:49.413580+0000 front 2024-02-05T08:26:49.413531+0000 (oldest deadline 2024-02-05T08:27:12.313452+0000) 2024-02-05T08:27:13.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:13 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:13.625+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:14.716 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:14 smithi157 podman[80334]: 2024-02-05 08:27:14.427938721 +0000 UTC m=+5.472629552 container died fa282b83539bd999e095fa3c5072a9431fef32e9bcbd16eabb5d385d886eb0e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:27:14.716 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:14 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:14.657+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:14.716 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:14 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:14.657+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:14.716 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:14 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:14.657+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:14.716 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:14 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:14.657+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-02-05T08:26:49.413580+0000 front 2024-02-05T08:26:49.413531+0000 (oldest deadline 2024-02-05T08:27:12.313452+0000) 2024-02-05T08:27:14.716 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:14 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:14.657+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:14.970 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:14 smithi157 podman[80334]: 2024-02-05 08:27:14.823465074 +0000 UTC m=+5.868155904 container cleanup fa282b83539bd999e095fa3c5072a9431fef32e9bcbd16eabb5d385d886eb0e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-02-05T08:27:14.970 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:14 smithi157 bash[80334]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6 2024-02-05T08:27:15.438 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:15 smithi157 podman[80356]: 2024-02-05 08:27:15.125053797 +0000 UTC m=+0.691346990 container remove fa282b83539bd999e095fa3c5072a9431fef32e9bcbd16eabb5d385d886eb0e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6, org.label-schema.build-date=20240131, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T08:27:15.702 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:15 smithi157 podman[80451]: 2024-02-05 08:27:15.34109407 +0000 UTC m=+0.017899400 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T08:27:15.703 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:15 smithi157 podman[80451]: 2024-02-05 08:27:15.459212548 +0000 UTC m=+0.136017915 container create e6287fd79700d10bf8d5c3c444c13b78f7fc6af791da415972901b9449f5bc72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_BRANCH=HEAD) 2024-02-05T08:27:15.703 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:15 smithi157 podman[80451]: 2024-02-05 08:27:15.703340513 +0000 UTC m=+0.380145838 container init e6287fd79700d10bf8d5c3c444c13b78f7fc6af791da415972901b9449f5bc72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_CLEAN=True) 2024-02-05T08:27:15.703 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:15 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:15.672+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:15.703 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:15 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:15.672+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:15.703 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:15 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:15.672+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:15.703 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:15 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:15.672+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-02-05T08:26:49.413580+0000 front 2024-02-05T08:26:49.413531+0000 (oldest deadline 2024-02-05T08:27:12.313452+0000) 2024-02-05T08:27:15.703 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:15 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:15.672+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:15.970 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:15 smithi157 podman[80451]: 2024-02-05 08:27:15.708642159 +0000 UTC m=+0.385447484 container start e6287fd79700d10bf8d5c3c444c13b78f7fc6af791da415972901b9449f5bc72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-05T08:27:15.970 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:15 smithi157 podman[80451]: 2024-02-05 08:27:15.811486708 +0000 UTC m=+0.488292043 container attach e6287fd79700d10bf8d5c3c444c13b78f7fc6af791da415972901b9449f5bc72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-05T08:27:16.289 INFO:journalctl@ceph.osd.6.smithi157.stdout:Feb 05 08:27:15 smithi157 podman[80451]: 2024-02-05 08:27:15.998567485 +0000 UTC m=+0.675372811 container died e6287fd79700d10bf8d5c3c444c13b78f7fc6af791da415972901b9449f5bc72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-02-05T08:27:16.582 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.6.service' 2024-02-05T08:27:16.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:27:16.623 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-05T08:27:16.623 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-05T08:27:16.623 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.7 2024-02-05T08:27:16.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:16 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:16.648+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:16.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:16 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:16.648+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:16.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:16 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:16.648+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:16.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:16 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:16.649+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-02-05T08:26:49.413580+0000 front 2024-02-05T08:26:49.413531+0000 (oldest deadline 2024-02-05T08:27:12.313452+0000) 2024-02-05T08:27:16.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:16 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:16.649+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:16.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:16 smithi157 systemd[1]: Stopping Ceph osd.7 for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:27:17.470 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:16 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:16.969+0000 7fdc21e5d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:27:17.470 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:16 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:16.969+0000 7fdc21e5d700 -1 osd.7 793 *** Got signal Terminated *** 2024-02-05T08:27:17.470 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:16 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:16.969+0000 7fdc21e5d700 -1 osd.7 793 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T08:27:17.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:17 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:17.679+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:17.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:17 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:17.679+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:17.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:17 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:17.679+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:17.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:17 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:17.679+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-02-05T08:26:49.413580+0000 front 2024-02-05T08:26:49.413531+0000 (oldest deadline 2024-02-05T08:27:12.313452+0000) 2024-02-05T08:27:17.971 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:17 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:17.680+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:18.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:18.664+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:18.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:18.664+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:18.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:18.664+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:18.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:18.664+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-02-05T08:26:49.413580+0000 front 2024-02-05T08:26:49.413531+0000 (oldest deadline 2024-02-05T08:27:12.313452+0000) 2024-02-05T08:27:18.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:18 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:18.664+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:19.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:19.653+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:19.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:19.653+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:19.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:19.653+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:19.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:19.653+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-02-05T08:26:49.413580+0000 front 2024-02-05T08:26:49.413531+0000 (oldest deadline 2024-02-05T08:27:12.313452+0000) 2024-02-05T08:27:19.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:19.653+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.157:6802 osd.4 since back 2024-02-05T08:26:56.414425+0000 front 2024-02-05T08:26:56.414586+0000 (oldest deadline 2024-02-05T08:27:18.714419+0000) 2024-02-05T08:27:19.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:19 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:19.653+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:20.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:20 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:20.623+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:20.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:20 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:20.623+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:20.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:20 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:20.623+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:20.971 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:20 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:20.623+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-02-05T08:26:49.413580+0000 front 2024-02-05T08:26:49.413531+0000 (oldest deadline 2024-02-05T08:27:12.313452+0000) 2024-02-05T08:27:20.971 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:20 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:20.623+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.157:6802 osd.4 since back 2024-02-05T08:26:56.414425+0000 front 2024-02-05T08:26:56.414586+0000 (oldest deadline 2024-02-05T08:27:18.714419+0000) 2024-02-05T08:27:20.971 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:20 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:20.624+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:21.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:21 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:21.648+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-02-05T08:26:26.110287+0000 front 2024-02-05T08:26:26.110183+0000 (oldest deadline 2024-02-05T08:26:52.010122+0000) 2024-02-05T08:27:21.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:21 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:21.648+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-02-05T08:26:32.511329+0000 front 2024-02-05T08:26:32.511397+0000 (oldest deadline 2024-02-05T08:26:56.611159+0000) 2024-02-05T08:27:21.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:21 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:21.648+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-02-05T08:26:41.312184+0000 front 2024-02-05T08:26:41.312222+0000 (oldest deadline 2024-02-05T08:27:03.012091+0000) 2024-02-05T08:27:21.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:21 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:21.648+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-02-05T08:26:49.413580+0000 front 2024-02-05T08:26:49.413531+0000 (oldest deadline 2024-02-05T08:27:12.313452+0000) 2024-02-05T08:27:21.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:21 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:21.648+0000 7fdc1a450700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.157:6802 osd.4 since back 2024-02-05T08:26:56.414425+0000 front 2024-02-05T08:26:56.414586+0000 (oldest deadline 2024-02-05T08:27:18.714419+0000) 2024-02-05T08:27:21.970 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:21 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7[57821]: 2024-02-05T08:27:21.648+0000 7fdc1a450700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24433.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-02-05T08:27:22.776 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:22 smithi157 podman[80582]: 2024-02-05 08:27:22.516450994 +0000 UTC m=+5.673670212 container died 59992933dbd3e4486b1efaf54654560eea0475f509d8357637c42fe87711a894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-05T08:27:23.142 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:22 smithi157 podman[80582]: 2024-02-05 08:27:22.866867012 +0000 UTC m=+6.024086231 container cleanup 59992933dbd3e4486b1efaf54654560eea0475f509d8357637c42fe87711a894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-05T08:27:23.142 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:22 smithi157 bash[80582]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7 2024-02-05T08:27:23.470 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:23 smithi157 podman[80604]: 2024-02-05 08:27:23.142878018 +0000 UTC m=+0.624603374 container remove 59992933dbd3e4486b1efaf54654560eea0475f509d8357637c42fe87711a894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True) 2024-02-05T08:27:23.761 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:23 smithi157 podman[80698]: 2024-02-05 08:27:23.476927549 +0000 UTC m=+0.111633951 container create 32737c868f5614871e6e5f56a60801c1edef2339f68d2a78f505a2b6cae98f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-05T08:27:23.762 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:23 smithi157 podman[80698]: 2024-02-05 08:27:23.383761767 +0000 UTC m=+0.018468131 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T08:27:23.762 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:23 smithi157 podman[80698]: 2024-02-05 08:27:23.679561324 +0000 UTC m=+0.314267688 container init 32737c868f5614871e6e5f56a60801c1edef2339f68d2a78f505a2b6cae98f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, ceph=True) 2024-02-05T08:27:23.762 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:23 smithi157 podman[80698]: 2024-02-05 08:27:23.685288046 +0000 UTC m=+0.319994412 container start 32737c868f5614871e6e5f56a60801c1edef2339f68d2a78f505a2b6cae98f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-05T08:27:23.762 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:23 smithi157 podman[80698]: 2024-02-05 08:27:23.762244264 +0000 UTC m=+0.396950635 container attach 32737c868f5614871e6e5f56a60801c1edef2339f68d2a78f505a2b6cae98f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7-deactivate, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD) 2024-02-05T08:27:24.220 INFO:journalctl@ceph.osd.7.smithi157.stdout:Feb 05 08:27:23 smithi157 podman[80698]: 2024-02-05 08:27:23.971543541 +0000 UTC m=+0.606249905 container died 32737c868f5614871e6e5f56a60801c1edef2339f68d2a78f505a2b6cae98f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240131, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T08:27:24.558 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@osd.7.service' 2024-02-05T08:27:24.597 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:27:24.597 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-05T08:27:24.597 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-05T08:27:24.597 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@rgw.foo.a 2024-02-05T08:27:24.899 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Feb 05 08:27:24 smithi131 systemd[1]: Stopping Ceph rgw.foo.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:27:25.166 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Feb 05 08:27:24 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-rgw-foo-a[70109]: 2024-02-05T08:27:24.898+0000 7fbd998ac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:27:25.166 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Feb 05 08:27:24 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-rgw-foo-a[70109]: 2024-02-05T08:27:24.898+0000 7fbda8117ac0 -1 shutting down 2024-02-05T08:27:35.267 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Feb 05 08:27:34 smithi131 bash[161515]: time="2024-02-05T08:27:34Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-05T08:27:35.267 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Feb 05 08:27:34 smithi131 podman[161515]: 2024-02-05 08:27:34.920615438 +0000 UTC m=+10.130069610 container died ecfde129a160d4460804706e652c3a342423b409c7bdd90876df00f2d0c852a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-rgw-foo-a, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-05T08:27:35.767 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Feb 05 08:27:35 smithi131 podman[161515]: 2024-02-05 08:27:35.495259424 +0000 UTC m=+10.704713595 container cleanup ecfde129a160d4460804706e652c3a342423b409c7bdd90876df00f2d0c852a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-rgw-foo-a, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-05T08:27:35.767 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Feb 05 08:27:35 smithi131 bash[161515]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-rgw-foo-a 2024-02-05T08:27:35.915 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@rgw.foo.a.service' 2024-02-05T08:27:35.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:27:35.955 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-05T08:27:35.955 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-05T08:27:35.955 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@prometheus.a 2024-02-05T08:27:36.261 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:27:36 smithi157 systemd[1]: Stopping Ceph prometheus.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:27:36.528 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:27:36 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T08:27:36.261Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-05T08:27:36.528 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:27:36 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T08:27:36.261Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-05T08:27:36.528 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:27:36 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T08:27:36.261Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-05T08:27:36.528 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:27:36 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T08:27:36.261Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-05T08:27:36.528 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:27:36 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T08:27:36.261Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-05T08:27:36.528 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:27:36 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T08:27:36.261Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-05T08:27:36.529 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:27:36 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T08:27:36.261Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-05T08:27:36.529 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:27:36 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T08:27:36.261Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-05T08:27:36.529 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:27:36 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T08:27:36.261Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-05T08:27:36.529 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:27:36 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T08:27:36.262Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-05T08:27:36.529 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:27:36 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T08:27:36.262Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-05T08:27:36.529 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:27:36 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a[68663]: ts=2024-02-05T08:27:36.263Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-05T08:27:36.529 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:27:36 smithi157 podman[80830]: 2024-02-05 08:27:36.271845162 +0000 UTC m=+0.121531084 container died 122617c83a3d69e5b6dd8b6abffe710a22c966f0fb0c7c582b9420e4ba494357 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T08:27:36.970 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:27:36 smithi157 podman[80830]: 2024-02-05 08:27:36.674995551 +0000 UTC m=+0.524681481 container cleanup 122617c83a3d69e5b6dd8b6abffe710a22c966f0fb0c7c582b9420e4ba494357 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T08:27:36.970 INFO:journalctl@ceph.prometheus.a.smithi157.stdout:Feb 05 08:27:36 smithi157 bash[80830]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-prometheus-a 2024-02-05T08:27:37.113 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@prometheus.a.service' 2024-02-05T08:27:37.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:27:37.152 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-05T08:27:37.152 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm rm-cluster --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-05T08:27:37.345 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T08:27:39.267 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 08:27:39 smithi131 systemd[1]: Stopping Ceph alertmanager.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:27:39.634 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 08:27:39 smithi131 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a[81153]: ts=2024-02-05T08:27:39.311Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-05T08:27:39.634 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 08:27:39 smithi131 podman[162034]: 2024-02-05 08:27:39.352810056 +0000 UTC m=+0.152323187 container died 6506f3a1485ab069cb8edd5e1c136c7abf6b71a4735388d6f43126040590e5e7 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T08:27:39.951 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 08:27:39 smithi131 podman[162034]: 2024-02-05 08:27:39.725655569 +0000 UTC m=+0.525168701 container cleanup 6506f3a1485ab069cb8edd5e1c136c7abf6b71a4735388d6f43126040590e5e7 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T08:27:39.951 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 08:27:39 smithi131 bash[162034]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a 2024-02-05T08:27:40.267 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 08:27:39 smithi131 podman[162053]: 2024-02-05 08:27:39.951907115 +0000 UTC m=+0.594278053 container remove 6506f3a1485ab069cb8edd5e1c136c7abf6b71a4735388d6f43126040590e5e7 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T08:27:40.267 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 08:27:40 smithi131 podman[162053]: 2024-02-05 08:27:40.019272692 +0000 UTC m=+0.661643631 volume remove 87e0ef917bc7979f9d8b271e17dae631254acb5ca824f673b0b26ca06dbc5e86 2024-02-05T08:27:40.267 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 08:27:40 smithi131 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-05T08:27:40.267 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 08:27:40 smithi131 systemd[1]: Stopped Ceph alertmanager.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T08:27:40.267 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Feb 05 08:27:40 smithi131 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@alertmanager.a.service: Consumed 2.499s CPU time. 2024-02-05T08:27:40.560 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 08:27:40 smithi131 systemd[1]: Stopping Ceph node-exporter.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:27:40.822 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 08:27:40 smithi131 podman[162173]: 2024-02-05 08:27:40.560514602 +0000 UTC m=+0.119419542 container died dfb42e8145217684c22cd354db2f0837d50d6939f1f8fa2bf7ce1961a6bb3895 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T08:27:41.189 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 08:27:40 smithi131 podman[162173]: 2024-02-05 08:27:40.92179154 +0000 UTC m=+0.480696475 container cleanup dfb42e8145217684c22cd354db2f0837d50d6939f1f8fa2bf7ce1961a6bb3895 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T08:27:41.189 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 08:27:40 smithi131 bash[162173]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a 2024-02-05T08:27:41.468 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 08:27:41 smithi131 podman[162192]: 2024-02-05 08:27:41.189523848 +0000 UTC m=+0.628795852 container remove dfb42e8145217684c22cd354db2f0837d50d6939f1f8fa2bf7ce1961a6bb3895 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T08:27:41.469 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 08:27:41 smithi131 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-05T08:27:41.469 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 08:27:41 smithi131 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-05T08:27:41.469 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 08:27:41 smithi131 systemd[1]: Stopped Ceph node-exporter.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T08:27:41.469 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Feb 05 08:27:41 smithi131 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@node-exporter.a.service: Consumed 5.191s CPU time. 2024-02-05T08:27:41.944 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm rm-cluster --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-05T08:27:42.136 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T08:27:43.471 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:27:43 smithi157 systemd[1]: Stopping Ceph iscsi.iscsi.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:27:43.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:27:43 smithi157 ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a[63388]: debug Shutdown received 2024-02-05T08:27:53.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:27:53 smithi157 bash[81260]: time="2024-02-05T08:27:53Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-02-05T08:27:53.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:27:53 smithi157 podman[81260]: 2024-02-05 08:27:53.676197998 +0000 UTC m=+10.145807580 container died 1327c0b7ca0e1f2db5da836c411462d3615912b6f9f2a61fb6483e86a640304a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a, GIT_CLEAN=True, io.buildah.version=1.33.2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS) 2024-02-05T08:27:54.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:27:54 smithi157 podman[81260]: 2024-02-05 08:27:54.069285393 +0000 UTC m=+10.538894962 container cleanup 1327c0b7ca0e1f2db5da836c411462d3615912b6f9f2a61fb6483e86a640304a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T08:27:54.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:27:54 smithi157 bash[81260]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a 2024-02-05T08:27:54.630 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:27:54 smithi157 podman[81285]: 2024-02-05 08:27:54.320564877 +0000 UTC m=+0.646196883 container remove 1327c0b7ca0e1f2db5da836c411462d3615912b6f9f2a61fb6483e86a640304a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-iscsi-iscsi-a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:27:54.630 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:27:54 smithi157 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-02-05T08:27:54.630 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:27:54 smithi157 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-02-05T08:27:54.630 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:27:54 smithi157 systemd[1]: Stopped Ceph iscsi.iscsi.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T08:27:54.630 INFO:journalctl@ceph.iscsi.iscsi.a.smithi157.stdout:Feb 05 08:27:54 smithi157 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@iscsi.iscsi.a.service: Consumed 4.419s CPU time. 2024-02-05T08:27:56.191 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 08:27:55 smithi157 systemd[1]: Stopping Ceph grafana.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:27:56.470 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 08:27:56 smithi157 podman[81563]: 2024-02-05 08:27:56.190924934 +0000 UTC m=+0.119737196 container died c01c9552d28da0c1a17a32a9d0ac238607a6de8c041d4dabc1c2d8ee5a5606c9 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a, release=239.1651231664, description=Ceph Grafana Container, version=8.5, io.openshift.expose-services=, io.buildah.version=1.32.0, maintainer=Paul Cuzner , vendor=Red Hat, Inc., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-04-29T11:48:11.055897, vcs-type=git, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, distribution-scope=public, name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=ubi8-container, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-02-05T08:27:56.909 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 08:27:56 smithi157 podman[81563]: 2024-02-05 08:27:56.50969727 +0000 UTC m=+0.438509533 container cleanup c01c9552d28da0c1a17a32a9d0ac238607a6de8c041d4dabc1c2d8ee5a5606c9 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a, maintainer=Paul Cuzner , version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=, com.redhat.component=ubi8-container, distribution-scope=public, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, release=239.1651231664, name=ubi8, io.k8s.display-name=Red Hat Universal Base Image 8, summary=Grafana Container configured for Ceph mgr/dashboard integration, vendor=Red Hat, Inc., description=Ceph Grafana Container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.buildah.version=1.32.0, architecture=x86_64) 2024-02-05T08:27:56.909 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 08:27:56 smithi157 bash[81563]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a 2024-02-05T08:27:56.909 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 08:27:56 smithi157 podman[81581]: 2024-02-05 08:27:56.719313415 +0000 UTC m=+0.527677585 container remove c01c9552d28da0c1a17a32a9d0ac238607a6de8c041d4dabc1c2d8ee5a5606c9 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-grafana-a, architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, build-date=2022-04-29T11:48:11.055897, com.redhat.component=ubi8-container, io.openshift.tags=base rhel8, io.buildah.version=1.32.0, io.openshift.expose-services=, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, release=239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, name=ubi8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=Ceph Grafana Container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, vcs-type=git, vendor=Red Hat, Inc., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner ) 2024-02-05T08:27:56.909 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 08:27:56 smithi157 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-02-05T08:27:57.220 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 08:27:57 smithi157 systemd[1]: Stopping Ceph node-exporter.b for 4018fbe8-c3fa-11ee-95b6-87774f69a715... 2024-02-05T08:27:57.220 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 08:27:56 smithi157 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-02-05T08:27:57.220 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 08:27:56 smithi157 systemd[1]: Stopped Ceph grafana.a for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T08:27:57.221 INFO:journalctl@ceph.grafana.a.smithi157.stdout:Feb 05 08:27:56 smithi157 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@grafana.a.service: Consumed 18.227s CPU time. 2024-02-05T08:27:57.663 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 08:27:57 smithi157 podman[81697]: 2024-02-05 08:27:57.36066953 +0000 UTC m=+0.111523876 container died 1c3496d1f04ded13db50fecb67d2f38aafff0f17bb63559704ae9a4faf0ed78e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T08:27:57.663 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 08:27:57 smithi157 podman[81697]: 2024-02-05 08:27:57.663679593 +0000 UTC m=+0.414533940 container cleanup 1c3496d1f04ded13db50fecb67d2f38aafff0f17bb63559704ae9a4faf0ed78e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T08:27:57.964 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 08:27:57 smithi157 bash[81697]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b 2024-02-05T08:27:57.965 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 08:27:57 smithi157 podman[81714]: 2024-02-05 08:27:57.847919874 +0000 UTC m=+0.491785174 container remove 1c3496d1f04ded13db50fecb67d2f38aafff0f17bb63559704ae9a4faf0ed78e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T08:27:57.965 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 08:27:57 smithi157 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-05T08:27:58.343 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 08:27:57 smithi157 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-05T08:27:58.343 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 08:27:57 smithi157 systemd[1]: Stopped Ceph node-exporter.b for 4018fbe8-c3fa-11ee-95b6-87774f69a715. 2024-02-05T08:27:58.343 INFO:journalctl@ceph.node-exporter.b.smithi157.stdout:Feb 05 08:27:57 smithi157 systemd[1]: ceph-4018fbe8-c3fa-11ee-95b6-87774f69a715@node-exporter.b.service: Consumed 5.314s CPU time. 2024-02-05T08:27:58.581 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-05T08:27:58.617 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-05T08:27:58.647 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-05T08:27:58.649 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546764/remote/smithi131/crash 2024-02-05T08:27:58.650 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/crash -- . 2024-02-05T08:27:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:tar: /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-05T08:27:58.753 INFO:teuthology.orchestra.run.smithi131.stderr:tar: Error is not recoverable: exiting now 2024-02-05T08:27:58.755 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546764/remote/smithi157/crash 2024-02-05T08:27:58.756 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/crash -- . 2024-02-05T08:27:58.795 INFO:teuthology.orchestra.run.smithi157.stderr:tar: /var/lib/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-05T08:27:58.795 INFO:teuthology.orchestra.run.smithi157.stderr:tar: Error is not recoverable: exiting now 2024-02-05T08:27:58.796 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-05T08:27:58.796 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-05T08:27:58.832 INFO:teuthology.orchestra.run.smithi131.stdout:2024-02-05T08:00:00.000195+0000 mon.a (mon.0) 2524 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-02-05T08:27:58.833 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-05T08:27:58.833 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[SEC\]' /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-05T08:27:58.909 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[ERR\]' /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-05T08:27:58.983 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[WRN\]' /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-05T08:27:59.054 INFO:teuthology.orchestra.run.smithi131.stdout:2024-02-05T08:00:00.000195+0000 mon.a (mon.0) 2524 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-02-05T08:27:59.055 INFO:tasks.cephadm:Compressing logs... 2024-02-05T08:27:59.055 DEBUG:teuthology.orchestra.run.smithi131:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-05T08:27:59.097 DEBUG:teuthology.orchestra.run.smithi157:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-05T08:27:59.123 INFO:teuthology.orchestra.run.smithi131.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-05T08:27:59.123 INFO:teuthology.orchestra.run.smithi131.stderr:: No such file or directory 2024-02-05T08:27:59.123 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-mon.a.log 2024-02-05T08:27:59.123 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.audit.log 2024-02-05T08:27:59.123 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-05T08:27:59.125 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.1.log 2024-02-05T08:27:59.125 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.3.log 2024-02-05T08:27:59.126 INFO:teuthology.orchestra.run.smithi157.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-05T08:27:59.126 INFO:teuthology.orchestra.run.smithi157.stderr:: No such file or directory 2024-02-05T08:27:59.126 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.6.log 2024-02-05T08:27:59.126 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.audit.log 2024-02-05T08:27:59.126 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.4.log 2024-02-05T08:27:59.128 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-mgr.y.log 2024-02-05T08:27:59.128 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.3.log: /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.0.log 2024-02-05T08:27:59.134 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.7.log 2024-02-05T08:27:59.134 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.5.log 2024-02-05T08:27:59.134 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.4.log: /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-mgr.x.log 2024-02-05T08:27:59.135 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/tcmu-runner.log 2024-02-05T08:27:59.135 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.cephadm.log 2024-02-05T08:27:59.135 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/tcmu-runner.log: 67.3% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/tcmu-runner.log.gz 2024-02-05T08:27:59.135 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.log 2024-02-05T08:27:59.135 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.cephadm.log: 81.5% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.cephadm.log.gz 2024-02-05T08:27:59.135 INFO:teuthology.orchestra.run.smithi157.stderr: 85.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-05T08:27:59.135 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-volume.log 2024-02-05T08:27:59.135 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-mon.b.log 2024-02-05T08:27:59.136 INFO:teuthology.orchestra.run.smithi131.stderr: 93.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-05T08:27:59.136 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.cephadm.log 2024-02-05T08:27:59.137 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-volume.log: 88.0% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.log.gz 2024-02-05T08:27:59.137 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-mon.b.log: 92.3% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.audit.log.gz 2024-02-05T08:27:59.138 INFO:teuthology.orchestra.run.smithi157.stderr: 92.6% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-mgr.x.log.gz 2024-02-05T08:27:59.140 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.log 2024-02-05T08:27:59.143 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.cephadm.log: 89.0% 95.3% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.audit.log.gz 2024-02-05T08:27:59.144 INFO:teuthology.orchestra.run.smithi157.stderr: 91.1% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-volume.log.gz 2024-02-05T08:27:59.153 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.2.log 2024-02-05T08:27:59.153 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.log: -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.cephadm.log.gz 2024-02-05T08:27:59.162 INFO:teuthology.orchestra.run.smithi131.stderr: 93.4% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph.log.gz 2024-02-05T08:27:59.163 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-volume.log 2024-02-05T08:27:59.172 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-mon.c.log 2024-02-05T08:27:59.189 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-volume.log: /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-mon.c.log: 94.1% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-05T08:27:59.200 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-volume.log.gz 2024-02-05T08:27:59.678 INFO:teuthology.orchestra.run.smithi131.stderr: 90.8% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-mgr.y.log.gz 2024-02-05T08:28:00.160 INFO:teuthology.orchestra.run.smithi157.stderr: 92.9% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-mon.b.log.gz 2024-02-05T08:28:00.772 INFO:teuthology.orchestra.run.smithi131.stderr: 92.5% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-mon.c.log.gz 2024-02-05T08:28:02.013 INFO:teuthology.orchestra.run.smithi131.stderr: 92.0% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-mon.a.log.gz 2024-02-05T08:28:06.780 INFO:teuthology.orchestra.run.smithi157.stderr: 93.8% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.6.log.gz 2024-02-05T08:28:06.877 INFO:teuthology.orchestra.run.smithi157.stderr: 93.9% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.5.log.gz 2024-02-05T08:28:07.035 INFO:teuthology.orchestra.run.smithi131.stderr: 93.9% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.0.log.gz 2024-02-05T08:28:07.324 INFO:teuthology.orchestra.run.smithi157.stderr: 93.9% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.7.log.gz 2024-02-05T08:28:07.674 INFO:teuthology.orchestra.run.smithi131.stderr: 93.9% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.2.log.gz 2024-02-05T08:28:07.874 INFO:teuthology.orchestra.run.smithi131.stderr: 94.0% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.3.log.gz 2024-02-05T08:28:08.187 INFO:teuthology.orchestra.run.smithi157.stderr: 94.0% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.4.log.gz 2024-02-05T08:28:08.193 INFO:teuthology.orchestra.run.smithi157.stderr: 2024-02-05T08:28:08.193 INFO:teuthology.orchestra.run.smithi157.stderr:real 0m9.078s 2024-02-05T08:28:08.193 INFO:teuthology.orchestra.run.smithi157.stderr:user 0m32.580s 2024-02-05T08:28:08.193 INFO:teuthology.orchestra.run.smithi157.stderr:sys 0m1.128s 2024-02-05T08:28:08.329 INFO:teuthology.orchestra.run.smithi131.stderr: 94.0% -- replaced with /var/log/ceph/4018fbe8-c3fa-11ee-95b6-87774f69a715/ceph-osd.1.log.gz 2024-02-05T08:28:08.331 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-02-05T08:28:08.331 INFO:teuthology.orchestra.run.smithi131.stderr:real 0m9.223s 2024-02-05T08:28:08.332 INFO:teuthology.orchestra.run.smithi131.stderr:user 0m38.090s 2024-02-05T08:28:08.332 INFO:teuthology.orchestra.run.smithi131.stderr:sys 0m1.336s 2024-02-05T08:28:08.332 INFO:tasks.cephadm:Archiving logs... 2024-02-05T08:28:08.332 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546764/remote/smithi131/log 2024-02-05T08:28:08.333 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-05T08:28:13.491 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546764/remote/smithi157/log 2024-02-05T08:28:13.492 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-05T08:28:18.094 INFO:tasks.cephadm:Removing cluster... 2024-02-05T08:28:18.094 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm rm-cluster --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 --force 2024-02-05T08:28:18.289 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T08:28:18.587 DEBUG:teuthology.orchestra.run.smithi157:> sudo cephadm rm-cluster --fsid 4018fbe8-c3fa-11ee-95b6-87774f69a715 --force 2024-02-05T08:28:18.769 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: 4018fbe8-c3fa-11ee-95b6-87774f69a715 2024-02-05T08:28:19.061 INFO:tasks.cephadm:Teardown complete 2024-02-05T08:28:19.061 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-05T08:28:19.077 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-05T08:28:19.078 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-05T08:28:19.081 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-05T08:28:19.162 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-05T08:28:19.218 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-05T08:28:19.219 DEBUG:teuthology.orchestra.run.smithi131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-05T08:28:19.222 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-05T08:28:19.234 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-02-05T08:28:19.236 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-02-05T08:28:19.238 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-05T08:28:19.238 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-02-05T08:28:19.238 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 7 +935us[ +935us] +/- 86ms 2024-02-05T08:28:19.238 INFO:teuthology.orchestra.run.smithi131.stdout:^* hv02.front.sepia.ceph.com 3 6 377 7 -657us[ -680us] +/- 71ms 2024-02-05T08:28:19.238 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 -151us[ -151us] +/- 80ms 2024-02-05T08:28:19.238 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T08:28:19.239 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-05T08:28:19.240 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-02-05T08:28:19.240 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 7 +931us[ +927us] +/- 86ms 2024-02-05T08:28:19.240 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv02.front.sepia.ceph.com 3 6 377 6 -660us[ -664us] +/- 71ms 2024-02-05T08:28:19.240 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -147us[ -151us] +/- 80ms 2024-02-05T08:28:19.240 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T08:28:19.241 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-05T08:28:19.253 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-05T08:28:19.253 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-05T08:28:19.317 DEBUG:teuthology.orchestra.run.smithi131:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-05T08:28:19.397 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-05T08:28:19.474 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-05T08:28:19.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T08:28:19.505 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-05T08:28:19.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T08:28:19.536 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-05T08:28:19.546 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-05T08:28:19.621 INFO:teuthology.task.internal:Duration was 3105.554420 seconds 2024-02-05T08:28:19.621 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-05T08:28:19.645 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-05T08:28:19.645 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-05T08:28:19.647 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-05T08:28:19.688 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-05T08:28:19.691 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-05T08:28:20.060 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-05T08:28:20.060 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi131.front.sepia.ceph.com 2024-02-05T08:28:20.061 DEBUG:teuthology.orchestra.run.smithi131:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-05T08:28:20.122 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-02-05T08:28:20.123 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-05T08:28:20.144 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-05T08:28:20.144 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-05T08:28:20.165 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-05T08:28:20.213 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-05T08:28:20.213 DEBUG:teuthology.orchestra.run.smithi131:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-05T08:28:20.233 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-05T08:28:20.688 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-05T08:28:20.698 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-05T08:28:20.699 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-05T08:28:20.725 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-05T08:28:20.753 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-05T08:28:20.765 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-05T08:28:20.768 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-05T08:28:20.791 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2024-02-05T08:28:20.819 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-02-05T08:28:20.845 DEBUG:teuthology.orchestra.run.smithi131:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-05T08:28:20.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T08:28:20.862 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-05T08:28:20.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T08:28:20.899 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-05T08:28:20.909 INFO:teuthology.task.internal:Transferring archived files... 2024-02-05T08:28:20.910 DEBUG:teuthology.misc:Transferring archived files from smithi131:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546764/remote/smithi131 2024-02-05T08:28:20.910 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-05T08:28:20.976 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546764/remote/smithi157 2024-02-05T08:28:20.976 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-05T08:28:21.015 INFO:teuthology.task.internal:Removing archive directory... 2024-02-05T08:28:21.015 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-05T08:28:21.017 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-05T08:28:21.063 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-05T08:28:21.078 INFO:teuthology.task.internal:Not uploading archives. 2024-02-05T08:28:21.079 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-05T08:28:21.094 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-05T08:28:21.094 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-05T08:28:21.097 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-05T08:28:21.108 INFO:teuthology.orchestra.run.smithi131.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 5 08:28 /home/ubuntu/cephtest 2024-02-05T08:28:21.118 INFO:teuthology.orchestra.run.smithi157.stdout: 265293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 5 08:28 /home/ubuntu/cephtest 2024-02-05T08:28:21.120 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-05T08:28:21.130 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-05T08:28:21.207 INFO:teuthology.nuke:Checking targets against current locks 2024-02-05T08:28:21.232 DEBUG:teuthology.nuke:shortname: smithi131 2024-02-05T08:28:21.232 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-05T08:28:21.256 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546764', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-05 07:26:53.390231', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDHqdPFifKEZTDIdgd+pGQ9OsOQQVTz8Qu79nVU1HggFfWWCt+N00mHMx5wMV+qaCuh/CLE3AjlTQGpFb/6SSEE='} 2024-02-05T08:28:21.260 DEBUG:teuthology.nuke:shortname: smithi157 2024-02-05T08:28:21.260 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-05T08:28:21.278 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546764', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-05 07:26:53.389042', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLl+NCAQSYeiV9rpCABJcuPqtPiDOjSZFuxQWYUIqg3f6zeHNOEe6AFKT8u7BgDJ1PDrQiXebd8ofmOBQLObPzQ='} 2024-02-05T08:28:21.303 INFO:teuthology.orchestra.console.smithi131:Power off 2024-02-05T08:28:21.303 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-05T08:28:21.326 DEBUG:teuthology.orchestra.console.smithi131:power off output: Chassis Power Control: Down/Off 2024-02-05T08:28:21.327 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T08:28:21.352 INFO:teuthology.orchestra.console.smithi157:Power off 2024-02-05T08:28:21.352 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-05T08:28:21.365 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2024-02-05T08:28:21.384 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2024-02-05T08:28:21.384 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T08:28:21.409 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2024-02-05T08:28:25.367 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T08:28:25.411 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T08:28:25.478 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2024-02-05T08:28:25.521 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2024-02-05T08:28:29.480 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T08:28:29.522 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T08:28:29.592 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off 2024-02-05T08:28:29.633 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2024-02-05T08:28:29.693 INFO:teuthology.orchestra.console.smithi131:Power off completed 2024-02-05T08:28:29.734 INFO:teuthology.orchestra.console.smithi157:Power off completed 2024-02-05T08:28:29.838 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} duration: 3105.5544197559357 failure_reason: '"2024-02-05T08:00:00.000195+0000 mon.a (mon.0) 2524 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled" in cluster log' flavor: default owner: scheduled_gabrioux@teuthology success: false 2024-02-05T08:28:29.839 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-05T08:28:29.915 INFO:teuthology.run:FAIL